From mika at async.caltech.edu  Wed Jan  1 11:48:36 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 01 Jan 2014 02:48:36 -0800
Subject: [M3devel] mklib not compiling
Message-ID: <20140101104836.E13C51A208E@async.async.caltech.edu>

Hi m3devel,

Trying to upgrade my compiler on AMD64_LINUX...

=== package /home/mika/cm3-cvs-anon/cm3/m3-sys/mklib ===
 +++ cm3 -build -DROOT='/home/mika/cm3-cvs-anon/cm3' $RARGS  && cm3 -ship $RARGS -DROOT='/home/mika/cm3-cvs-anon/cm3'  +++
--- building in AMD64_LINUX ---

ignoring ../src/m3overrides

new source -> compiling Main.m3
"../src/Main.m3", line 659: incompatible types (b)
"../src/Main.m3", line 660: incompatible types (b)
"../src/Main.m3", line 661: incompatible types (b)
"../src/Main.m3", line 662: incompatible types (b)
"../src/Main.m3", line 663: incompatible types (b)
"../src/Main.m3", line 664: incompatible types (b)
"../src/Main.m3", line 665: incompatible types (b)
7 errors encountered
compilation failed => not building program "mklib"
Fatal Error: package build failed

I'm at the head...

    Mika


From mika at async.caltech.edu  Thu Jan  2 17:34:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 08:34:10 -0800
Subject: [M3devel] building the CVS head---how???
Message-ID: <20140102163410.757DB1A2080@async.async.caltech.edu>


So here is the situation.

New machine, new problems.  

AMD64_LINUX.  I want to try the latest compiler, because there are issues with what I have.

What I have is the following:

Critical Mass Modula-3 version 5.8.6
  last updated: 2010-04-11
  compiled: 2010-07-12 20:10:34
  configuration: /home/mika/cm3.amd64-linux/bin/cm3.cfg
  host: AMD64_LINUX
  target: AMD64_LINUX

I think the boot archives on the CM3 web site are about the same age.

I try 

./boot1.py AMD64_LINUX

and I get

"../src/win32/WinSock.i3", line 662: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 665: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 688: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 691: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 694: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 697: unsupported language or calling convention (PASCAL)
34 errors encountered
new exporters -> recompiling WinVer.i3
"../src/win32/WinVer.i3", line 158: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 168: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 180: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 190: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 203: unsupported language or calling convention (APIENTRY)

(etc)

all right, it's broken, something needs to be fixed.

WHAT IS NOT ALL RIGHT:

At this point my regular cm3.cfg is messed up.  If I had been silly enough to try to bootstrap
a new compiler (for a different machine) without first saving my old CM3 installation, I'd
be screwed.

But from reading the mailing list logs, I am not sure if it is even possible to bootstrap
the head from the older installation.  How am I supposed to get a new compiler working?
Build it on my Raspberry Pi (which has a working, recent CM3 on it)?  (After backing up
my old installation, of course.)

And what's this?  Bad python?

(119)truffles:~/cm3-cvs-anon/cm3/scripts/python>./boot1.py AMD64_LINUX
Traceback (most recent call last):
  File "./boot1.py", line 5, in <module>
    import pylib
  File "/big/home2/mika/2/cm3-cvs/cm3/scripts/python/pylib.py", line 587, in <module>
    for a in os.popen(CM3 + " -version 2>" + DevNull):
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
(120)truffles:~/cm3-cvs-anon/cm3/scripts/python>python --version
Python 2.7.3

     Mika
     Mika


From mika at async.caltech.edu  Fri Jan  3 00:15:37 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 15:15:37 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
Message-ID: <20140102231537.B953A1A2080@async.async.caltech.edu>


So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing

./boot1.py c AMD64_LINUX

I get:

new source -> compiling PklActionSeq.i3
new source -> compiling ConvertPacking.m3
"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)


***
*** runtime error:
***    <*ASSERT*> failed.
***    file "../src/M3C.m3", line 5890
***

Aborted
 *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***


   5887 PROCEDURE shift_right(self: T; type: IType) =
   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
   5889 BEGIN
   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
   5891     shift_left_or_right(self, type, "shift_right", ">>");
   5892 END shift_right;

...

Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...

     Mika


From mika at async.caltech.edu  Fri Jan  3 06:12:52 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:12:52 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
In-Reply-To: <20140102174632.3169C41A@resin13.mta.everyone.net>
References: <20140102174632.3169C41A@resin13.mta.everyone.net>
Message-ID: <20140103051252.88FC61A2080@async.async.caltech.edu>

Well I managed to get it to work using the following roundabout procedure:

Since the only archives I could find on elegosoft for AMD64_LINUX were ancient,
I installed VirtualBox and FreeBSD/amd64 inside that.  FreeBSD/amd64 for some
reason has very recent archives (of the old "cminstall" kind).

Downloaded the FreeBSD archive, cminstalled it, and ran upgrade.py on the latest
sources.

Had to remove some code from ConvertPacking.m3 because Jay's C-backend can't compile
that for some reason.

ran boot1.py, which gave me a boot archive.  Note---if you've done this with boot1.py "c"
before, you have to delete all the derived dirs and start over, or you get a mess.

Moved the boot archive to the Linux machine.

Resurrected ConvertPacking.m3

Ran boot2.sh (or equiv) until I got everything built.

That worked.

So... some observations: 

.. I have a very recent build on ARM_LINUX, but wasn't able to cross-compile.  Word size
problem?

.. I now have three very recent builds: ARM_LINUX, AMD64_FREEBSD, and AMD64_LINUX.  What
precisely do I have to do to make boot archives out of these so other people might be spared
this painful process?  Very few archives on the elego site are up to date.  Some NT stuff and
some FreeBSD stuff.  Nothing else that I can see.

    Mika

"Rodney Bates" writes:
>
>
>-Rodney Bates
>
>--- mika at async.caltech.edu wrote:
>
>From: mika at async.caltech.edu
>To: m3devel at elegosoft.com
>Subject: [M3devel] booting for AMD64_LINUX, continued...
>Date: Thu, 02 Jan 2014 15:15:37 -0800
>
>
>So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing
>
>./boot1.py c AMD64_LINUX
>
>I get:
>
>new source -> compiling PklActionSeq.i3
>new source -> compiling ConvertPacking.m3
>"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
>"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)
>
>
>***
>*** runtime error:
>***    <*ASSERT*> failed.
>***    file "../src/M3C.m3", line 5890
>***
>
>Aborted
> *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***
>
>
>   5887 PROCEDURE shift_right(self: T; type: IType) =
>   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
>   5889 BEGIN
>   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
>   5891     shift_left_or_right(self, type, "shift_right", ">>");
>   5892 END shift_right;
>
>...
>
>Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...
>
>I have one, but am out of the country right now and have no access to it.  
>I am waiting as I write to board the first airplane home, so will try to
>tar one up for you when I get there. 
>
>
>     Mika
>


From mika at async.caltech.edu  Fri Jan  3 06:57:18 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:57:18 -0800
Subject: [M3devel] pthreads still broken
Message-ID: <20140103055718.B12821A2080@async.async.caltech.edu>

On my new AMD64_LINUX build...

The thread tester still crashes.

First, it prints a couple of extraneous diagnostics.  These look like
they come from the threading code?

Then it segfaults.

(141)truffles:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX>gdb threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest...done.
(gdb) run 
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x2aaaab6f8700 (LWP 23996)]
[New Thread 0x2aaaab8f9700 (LWP 23997)]
[New Thread 0x2aaaabafa700 (LWP 23998)]
done
Creating fork threads...[New Thread 0x2aaaabcfb700 (LWP 23999)]
[New Thread 0x2aaaabefc700 (LWP 24000)]
[New Thread 0x2aaaac0fd700 (LWP 24001)]
done
Creating alloc threads...[New Thread 0x2aaaac2fe700 (LWP 24002)]
[New Thread 0x2aaaac4ff700 (LWP 24003)]
[New Thread 0x2aaaac700700 (LWP 24004)]
done
Creating lock threads...[New Thread 0x2aaaac901700 (LWP 24005)]
[New Thread 0x2aaaacb02700 (LWP 24006)]
[New Thread 0x2aaaacd03700 (LWP 24007)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x2aaaac901700 (LWP 24005)]
0x00002aaaaaf5b64b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x2aaaab6f8700 (LWP 23996)]
0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
44          IF (rd.buff = NIL) THEN
(gdb) where
#0  0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
#1  0x0000000000416d3a in FileRd__Open (M3_Bd56fi_p=<error reading variable>) at ../src/rw/FileRd.m3:16
#2  0x0000000000405424 in Main__RApply (M3_AP7a1g_cl=<error reading variable>) at ../src/Main.m3:182
#3  0x000000000044c23b in ThreadPThread__RunThread (M3_DMxDjQ_me=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:449
#4  0x000000000044bef8 in ThreadPThread__ThreadBase (M3_AJWxb1_param=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#5  0x00002aaaaaf56b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00002aaaab246a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x0000000000000000 in ?? ()
(gdb) 

The reason I even ran it was because I found that running the parallel compiler backend was unstable. 
  
Do user threads work on AMD64_LINUX?

     Mika


From mika at async.caltech.edu  Fri Jan  3 17:13:43 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 08:13:43 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
Message-ID: <20140103161343.F25291A2080@async.async.caltech.edu>

(78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LINUX/threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest...done.
(gdb) handle SIGILL 
Signal        Stop      Print   Pass to program Description
SIGILL        Yes       Yes     Yes             Illegal instruction
(gdb) handle SIGILL  nostop noprint
Signal        Stop      Print   Pass to program Description
SIGILL        No        No      Yes             Illegal instruction
(gdb) run
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
[New Thread 0x40bf4450 (LWP 2942)]
[New Thread 0x40df4450 (LWP 2943)]
done
Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
[New Thread 0x411f4450 (LWP 2945)]
[New Thread 0x413f4450 (LWP 2946)]
done
Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
[New Thread 0x417f4450 (LWP 2948)]
[New Thread 0x419f4450 (LWP 2949)]
done
Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
[New Thread 0x41df4450 (LWP 2951)]
[New Thread 0x41ff4450 (LWP 2952)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x41ff4450 (LWP 2952)]
0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "../src/runtime/common/RTCollector.m3", line 418
***


Program received signal SIGABRT, Aborted.
[Switching to Thread 0x40ff4450 (LWP 2944)]
0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) where
#0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
#2  0x00000000 in ?? ()
(gdb) 

It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
ARM_LINUX but would be surprised if it didn't work.

Yes the pthreads seem "more reliable" than they used to be, but I would
never use them in any sort of mission-critical application.  I use
Modula-3 programs for (among other things) financial transactions and
real-time motor controllers...

    Mika


From mika at async.caltech.edu  Fri Jan  3 18:19:32 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 09:19:32 -0800
Subject: [M3devel] AMD64_LINUX update...
Message-ID: <20140103171932.F28D61A2080@async.async.caltech.edu>


Hi again m3devel,

With the head and user threads, after bootstrapping via AMD64_FREEBSD,
I'm getting everything (apparently) working EXCEPT one thing:

stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
into an infinite loop, looks like it's happening on the stack because I
run out of memory rather quickly, on certain types local to my own system.

Has anyone seen this before?  I've never seen it on any 32-bit system
but I have a vague memory of seeing it on 64-bit systems.

Will debug over the weekend...

I'd be grateful for any input from the list on the pthreads issues, though.

     Mika


From mika at async.caltech.edu  Sat Jan  4 19:55:11 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 10:55:11 -0800
Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX
Message-ID: <20140104185511.9000A1A2080@async.async.caltech.edu>


I have this program that generates Modula-3 code from XML.

It uses Wr.PutText a lot, on small strings.

It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
Core i7 4960X.

(176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w

(29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w

I believe the issue is that Text.m3 is making a ton of system calls!

Here's a typical traceback:

(gdb) where
#0  0x00002aaaaaacb770 in ?? ()
#1  0x00002aaaaaacba1b in gettimeofday ()
#2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
#3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
#4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
#5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
#6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>, 
    M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
#7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
#8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
#9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>, 
    M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
#10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
#11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
#12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>, 
    M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
#13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
#14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>, 
    M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>, 
    M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>, 
    M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
#15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
#16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
(gdb) 

I believe all the TextStats operations are killing the performance.  Can we remove them?

Here is SetChars:

PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
  BEGIN
    TextStats.NoteGround (TextStats.Op.SetChars);
    TextStats.NoteGround (TextStats.Op.get_chars);
    t.get_chars (a, start);
    TextStats.NoteFinished (TextStats.Op.get_chars);
    TextStats.NoteFinished (TextStats.Op.SetChars);
  END SetChars;

I'm not sure how best to do this.  Removing them without killing functionality might not be
easy.  Looks like a case for conditional compilation.

Maybe generate the .m3s from some other source, with and without the TextStats, depending on
a setting in m3makefile?

Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.  

     Mika



From dragisha at m3w.org  Sat Jan  4 22:04:22 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:04:22 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140103171932.F28D61A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
Message-ID: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>

Hi Mika,

I have this for LINUXLIBC6, and I am using it for remake of HEAD: cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz

Can you build and share cm3-bin-core for AMD64_LINUX?

TIA,
dd

On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:

> 
> Hi again m3devel,
> 
> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
> I'm getting everything (apparently) working EXCEPT one thing:
> 
> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
> into an infinite loop, looks like it's happening on the stack because I
> run out of memory rather quickly, on certain types local to my own system.
> 
> Has anyone seen this before?  I've never seen it on any 32-bit system
> but I have a vague memory of seeing it on 64-bit systems.
> 
> Will debug over the weekend...
> 
> I'd be grateful for any input from the list on the pthreads issues, though.
> 
>     Mika

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment.sig>

From mika at async.caltech.edu  Sat Jan  4 22:12:00 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:12:00 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
Message-ID: <20140104211200.A80571A2080@async.async.caltech.edu>

Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?

I can do this for both AMD64_LINUX (user threads)
and ARM_LINUX (pthreads)

I might still have AMD64_LINUX (pthreads)

Also have AMD64_FREEBSD (pthreads) from the head.

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=us-ascii
>
>Hi Mika,
>
>I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>
>Can you build and share cm3-bin-core for AMD64_LINUX?
>
>TIA,
>dd
>
>On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>
>>=20
>> Hi again m3devel,
>>=20
>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>> I'm getting everything (apparently) working EXCEPT one thing:
>>=20
>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>> into an infinite loop, looks like it's happening on the stack because =
>I
>> run out of memory rather quickly, on certain types local to my own =
>system.
>>=20
>> Has anyone seen this before?  I've never seen it on any 32-bit system
>> but I have a vague memory of seeing it on 64-bit systems.
>>=20
>> Will debug over the weekend...
>>=20
>> I'd be grateful for any input from the list on the pthreads issues, =
>though.
>>=20
>>     Mika
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/html;
>	charset=us-ascii
>
><html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; font-family: Candara; =
>border-spacing: 0px;"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>space; -webkit-line-break: after-white-space; ">dd</div><div =
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>-webkit-line-break: after-white-space; =
>"><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>at 18:19, <a =
>href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>infinite loop, looks like it's happening on the stack because I<br>run =
>out of memory rather quickly, on certain types local to my own =
>system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>for any input from the list on the pthreads issues, though.<br><br> =
>&nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>=
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Transfer-Encoding: 7bit
>Content-Disposition: attachment;
>	filename=signature.asc
>Content-Type: application/pgp-signature;
>	name=signature.asc
>Content-Description: Message signed with OpenPGP using GPGMail
>
>-----BEGIN PGP SIGNATURE-----
>Comment: GPGTools - http://gpgtools.org
>
>iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>=sMKJ
>-----END PGP SIGNATURE-----
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--


From mika at async.caltech.edu  Sat Jan  4 22:15:51 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:15:51 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
In-Reply-To: <EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
References: <20140103161343.F25291A2080@async.async.caltech.edu>
	<EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
Message-ID: <20140104211551.54F481A2080@async.async.caltech.edu>

Antony Hosking writes:
>I wonder of the redzone size for AMD 64 stack frames is not set properly.=20=
>

Tony,

Are you talking about the difference between the ARM and AMD64 results?

The AMD64 is also a machine with 12 logical (HT) processors whereas the
ARM is a single processor.  That could make a difference in how it fails.

By the way, would you know:  The following output on the amd64:

Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Is that an assertion failing in the pthreads library?  My program isn't
printing this stuff: it's coming from ThreadPThread or some such code.

     Mika

>
>Sent from my iPhone
>
>> On Jan 3, 2014, at 10:13 AM, mika at async.caltech.edu wrote:
>>=20
>> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LIN=
>UX/threadtest
>> GNU gdb (GDB) 7.4.1-debian
>> Copyright (C) 2012 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.ht=
>ml>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"=
>
>> and "show warranty" for details.
>> This GDB was configured as "arm-linux-gnueabihf".
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>...
>> Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/th=
>read/ARM_LINUX/threadtest...done.
>> (gdb) handle SIGILL=20
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        Yes       Yes     Yes             Illegal instruction
>> (gdb) handle SIGILL  nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        No        No      Yes             Illegal instruction
>> (gdb) run
>> Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/threa=
>d/ARM_LINUX/threadtest=20
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.=
>1".
>> Writing file...done
>> Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
>> [New Thread 0x40bf4450 (LWP 2942)]
>> [New Thread 0x40df4450 (LWP 2943)]
>> done
>> Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
>> [New Thread 0x411f4450 (LWP 2945)]
>> [New Thread 0x413f4450 (LWP 2946)]
>> done
>> Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
>> [New Thread 0x417f4450 (LWP 2948)]
>> [New Thread 0x419f4450 (LWP 2949)]
>> done
>> Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
>> [New Thread 0x41df4450 (LWP 2951)]
>> [New Thread 0x41ff4450 (LWP 2952)]
>> done
>> running...printing oldest/median age/newest
>>=20
>> Program received signal SIG64, Real-time event 64.
>> [Switching to Thread 0x41ff4450 (LWP 2952)]
>> 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnu=
>eabihf/libpthread.so.0
>> (gdb) handle SIG64 nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIG64         No        No      Yes             Real-time event 64
>> (gdb) cont
>> Continuing.
>> .
>>=20
>> ***
>> *** runtime error:
>> ***    An array subscript was out of range.
>> ***    file "../src/runtime/common/RTCollector.m3", line 418
>> ***
>>=20
>>=20
>> Program received signal SIGABRT, Aborted.
>> [Switching to Thread 0x40ff4450 (LWP 2944)]
>> 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> (gdb) where
>> #0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> #1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
>> #2  0x00000000 in ?? ()
>> (gdb)=20
>>=20
>> It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
>> ARM_LINUX but would be surprised if it didn't work.
>>=20
>> Yes the pthreads seem "more reliable" than they used to be, but I would
>> never use them in any sort of mission-critical application.  I use
>> Modula-3 programs for (among other things) financial transactions and
>> real-time motor controllers...
>>=20
>>    Mika


From dragisha at m3w.org  Sat Jan  4 22:52:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:52:32 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104211200.A80571A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
Message-ID: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>

-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*

there they are. I don?t do this too often, but both do-cm3-core and do-cm3-min will do for what I need.

TIA,
dd

On 04 Jan 2014, at 22:12, mika at async.caltech.edu wrote:

> Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?
> 
> I can do this for both AMD64_LINUX (user threads)
> and ARM_LINUX (pthreads)
> 
> I might still have AMD64_LINUX (pthreads)
> 
> Also have AMD64_FREEBSD (pthreads) from the head.
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=us-ascii
>> 
>> Hi Mika,
>> 
>> I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>> cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>> 
>> Can you build and share cm3-bin-core for AMD64_LINUX?
>> 
>> TIA,
>> dd
>> 
>> On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> Hi again m3devel,
>>> =20
>>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>>> I'm getting everything (apparently) working EXCEPT one thing:
>>> =20
>>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>>> into an infinite loop, looks like it's happening on the stack because =
>> I
>>> run out of memory rather quickly, on certain types local to my own =
>> system.
>>> =20
>>> Has anyone seen this before?  I've never seen it on any 32-bit system
>>> but I have a vague memory of seeing it on 64-bit systems.
>>> =20
>>> Will debug over the weekend...
>>> =20
>>> I'd be grateful for any input from the list on the pthreads issues, =
>> though.
>>> =20
>>>    Mika
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/html;
>> 	charset=us-ascii
>> 
>> <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>> charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>> Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>> for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>> 8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>> orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>> share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>> apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; font-family: Candara; =
>> border-spacing: 0px;"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>> ">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>> space; -webkit-line-break: after-white-space; ">dd</div><div =
>> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>> -webkit-line-break: after-white-space; =
>> "><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>> at 18:19, <a =
>> href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>> wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>> type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>> threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>> everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>> derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>> infinite loop, looks like it's happening on the stack because I<br>run =
>> out of memory rather quickly, on certain types local to my own =
>> system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>> any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>> systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>> for any input from the list on the pthreads issues, though.<br><br> =
>> &nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>> =
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Transfer-Encoding: 7bit
>> Content-Disposition: attachment;
>> 	filename=signature.asc
>> Content-Type: application/pgp-signature;
>> 	name=signature.asc
>> Content-Description: Message signed with OpenPGP using GPGMail
>> 
>> -----BEGIN PGP SIGNATURE-----
>> Comment: GPGTools - http://gpgtools.org
>> 
>> iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>> taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>> zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>> 0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>> hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>> 1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>> =sMKJ
>> -----END PGP SIGNATURE-----
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment.sig>

From mika at async.caltech.edu  Sat Jan  4 22:58:53 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:58:53 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
Message-ID: <20140104215853.09F0D1A2080@async.async.caltech.edu>

Hi dd,

I already have everything built.  

What I'm asking is how I go from what I have to a tar file.  What is
included?  Is there a script for that?

I can just give you the full binary directory or the full cm3 system with
sources, but I suspect there is a reduced set you would prefer?

BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
scripts in python work fine.  If you already have cm3 from the head
it ought to be possible to cross-build.  But I did have problems going from
32 to 64 bits...

     Mika

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>
>
>--Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=windows-1252
>
>-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>
>there they are. I don=92t do this too often, but both do-cm3-core and =
>do-cm3-min will do for what I need.
>
>TIA,
>dd


From dragisha at m3w.org  Sat Jan  4 23:10:04 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:10:04 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104215853.09F0D1A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
Message-ID: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>

do-cm3-bin will build a tar.gz, I think. And it is not too big.

I did not try boot*.py as this box I would like to build for is remote and all? I have src.rpm and I only need this cm3-min? to make it work and build itself on remote box.

TIA,
dd

On 04 Jan 2014, at 22:58, mika at async.caltech.edu wrote:

> Hi dd,
> 
> I already have everything built.  
> 
> What I'm asking is how I go from what I have to a tar file.  What is
> included?  Is there a script for that?
> 
> I can just give you the full binary directory or the full cm3 system with
> sources, but I suspect there is a reduced set you would prefer?
> 
> BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
> scripts in python work fine.  If you already have cm3 from the head
> it ought to be possible to cross-build.  But I did have problems going from
> 32 to 64 bits...
> 
>     Mika
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>> 
>> 
>> --Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=windows-1252
>> 
>> -rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>> -rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>> -rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>> -rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>> -rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>> -rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>> -rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>> 
>> there they are. I don=92t do this too often, but both do-cm3-core and =
>> do-cm3-min will do for what I need.
>> 
>> TIA,
>> dd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment.sig>

From mika at async.caltech.edu  Sat Jan  4 23:23:21 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:23:21 -0800
Subject: [M3devel] can't write to repository
Message-ID: <20140104222321.B53051A2080@async.async.caltech.edu>

Hi m3devel,

I can't seem to write to the CVS repository.  I get:

(62)rover:~/cm3-writable/cm3/scripts>cvs up -d .
mika at birch.elegosoft.com's password: 

Has my account been removed?  I don't think I've changed keys.

In any case, could someone please commit the following change:

~/cm3-cvs-anon/cm3/scripts>cvs diff sysinfo.sh
Index: sysinfo.sh
===================================================================
RCS file: /usr/cvs/cm3/scripts/sysinfo.sh,v
retrieving revision 1.109
diff -r1.109 sysinfo.sh
273a274
>         arm*)    CM3_TARGET=ARM_LINUX;;

    Mika


From mika at async.caltech.edu  Sat Jan  4 23:48:13 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:48:13 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
Message-ID: <20140104224813.34D171A2080@async.async.caltech.edu>


I think the script you meant was make-bin-dist-min.sh

The results are at:

http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz

Please upload this to the CM3 website if it looks OK to you and you know how to upload it.

It's set up to use user threads.

    Mika



From dragisha at m3w.org  Sat Jan  4 23:57:55 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:57:55 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104224813.34D171A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
Message-ID: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>

I am using it to bootstrap my rpm, did you try copying to: /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

I will make my rpm tomorrow, and try uploading all of it then.

Thanks,
dd



On 04 Jan 2014, at 23:48, mika at async.caltech.edu wrote:

> 
> I think the script you meant was make-bin-dist-min.sh
> 
> The results are at:
> 
> http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz
> 
> Please upload this to the CM3 website if it looks OK to you and you know how to upload it.
> 
> It's set up to use user threads.
> 
>    Mika
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment.sig>

From mika at async.caltech.edu  Sat Jan  4 23:59:24 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:59:24 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
Message-ID: <20140104225924.858A21A2080@async.async.caltech.edu>

>I am using it to bootstrap my rpm, did you try copying to: =
>/var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

Per my email of a few minutes ago, I am unable to get birch to accept
my login credentials...


From dragisha at m3w.org  Sun Jan  5 01:44:17 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 01:44:17 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104225924.858A21A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
Message-ID: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>

Copied, and did ?a sensible thing?, ran update script? It did not create as complete index.html as it was before. I will look into it tomorrow, it is too late for me now to believe my further fixing would not break more things :).


On 04 Jan 2014, at 23:59, mika at async.caltech.edu wrote:

>> I am using it to bootstrap my rpm, did you try copying to: =
>> /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?
> 
> Per my email of a few minutes ago, I am unable to get birch to accept
> my login credentials...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/841337b6/attachment.sig>

From mika at async.caltech.edu  Sun Jan  5 03:50:44 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 18:50:44 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
Message-ID: <20140105025044.9FC551A2080@async.async.caltech.edu>


Another favor?

Could you upload to CM3 the file at

http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz 

If there is a "notes" section, you can note that the distribution works on 

1.  Raspberry Pi running Raspbian Wheezy
Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l

2.  BeagleBone Black running (BeagleBoard)Ubuntu 13.10 (GNU/Linux 3.8.13-bone28 armv7l)
Linux arm 3.8.13-bone28 #1 SMP Fri Sep 13 01:11:14 UTC 2013 armv7l armv7l armv7l GNU/Linux

(Anyone reading this mail who wants a copy can of course also use the link to rover above.)

     Mika


From dragisha at m3w.org  Sun Jan  5 21:15:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 21:15:23 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140105025044.9FC551A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
Message-ID: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>

Also on http://opencm3.net/snaps/


On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:

> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment.sig>

From dragisha at m3w.org  Mon Jan  6 06:02:36 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 6 Jan 2014 06:02:36 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
	<6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
Message-ID: <9C223752-0979-4BC0-84C7-C8F83B9BF01B@m3w.org>

There is hourly run script for user m3 which updates index file in this folder. It keeps deleteng your AMD_LINUX snapshot, apparently after indexing it.

ARM_LINUX one is indexed and left in peace. I will look into this issue in few days.

dd

On 05 Jan 2014, at 21:15, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Also on http://opencm3.net/snaps/
> 
> 
> On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:
> 
>> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment.sig>

From rodney_bates at lcwb.coop  Mon Jan  6 20:41:53 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 06 Jan 2014 13:41:53 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CB0701.9030307@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.
>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.

Try the fix I just checked in to TextClass.m3.  It was already set up with such a boolean flag
(TextClass.CollectStats, set FALSE), but in one of the instrumentation procedures, two system
calls were outside the test thereof.  Hopefully, this will give a good bit of relief for now,
with a simple fix.

>
>       Mika
>
>



From rodney_bates at lcwb.coop  Thu Jan  9 22:01:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 09 Jan 2014 15:01:55 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CF0E43.2030701@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.

Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
only needed for performance study of the algorithms, which likely won't be done often.

>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
>
>       Mika
>
>



From jay.krell at cornell.edu  Fri Jan 10 09:22:25 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 10 Jan 2014 08:22:25 +0000
Subject: [M3devel] performance problems in CM3 Text operations on
 AMD64_LINUX
In-Reply-To: <52CF0E43.2030701@lcwb.coop>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
Message-ID: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>

Ok either, way, but here is another point.

In some of our code we do:

CONST DEBUG = FALSE;
IF FALSE THEN   xxEND;


It will tend to be optimized.Furthermore, syscalls -- even if not optimized,IF FALSE is going to be way faster than a syscall.

Also, gettimeofday shouldn't necessarily be a syscall.

But/and there are/should-be faster ways of getting time for benchmarking.

On Windows, GetTickCount and GetTickCount64 just read a value out of memory.Even GetSystemTimeAsFileTime is just a read of a 64bit value from memory.(The memory is read only, at the same address in all processes, and updated bythe kernel.)x86 has rdtsc, a 64bit hardware cycle counter readable in one instruction.



We should consider researching analogs to rdtsc across mips/powerpc/sparc/arm andprovide something in m3core. I think.Add it to the backend interface likely, for inlining.or maybe it is worth a function call, then just use #ifdefed C.


Maybe Posix provides something that tends to be like this?

Mika, how did you find this?Control-c every so often and look at callstack?Or a lot of stepping?

 - Jay

: Thu, 9 Jan 2014 15:01:55 -0600
> From: rodney_bates at lcwb.coop
> To: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] performance problems in CM3 Text operations on	AMD64_LINUX
> 
> 
> 
> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> > I have this program that generates Modula-3 code from XML.
> >
> > It uses Wr.PutText a lot, on small strings.
> >
> > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> > Core i7 4960X.
> >
> > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
> >
> > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
> >
> > I believe the issue is that Text.m3 is making a ton of system calls!
> >
> > Here's a typical traceback:
> >
> > (gdb) where
> > #0  0x00002aaaaaacb770 in ?? ()
> > #1  0x00002aaaaaacba1b in gettimeofday ()
> > #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> > #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> > #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> > #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> > #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
> >      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> > #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> > #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> > #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
> >      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
> >      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
> >      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
> >      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
> >      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> > (gdb)
> >
> > I believe all the TextStats operations are killing the performance.  Can we remove them?
> >
> > Here is SetChars:
> >
> > PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
> >    BEGIN
> >      TextStats.NoteGround (TextStats.Op.SetChars);
> >      TextStats.NoteGround (TextStats.Op.get_chars);
> >      t.get_chars (a, start);
> >      TextStats.NoteFinished (TextStats.Op.get_chars);
> >      TextStats.NoteFinished (TextStats.Op.SetChars);
> >    END SetChars;
> >
> > I'm not sure how best to do this.  Removing them without killing functionality might not be
> > easy.  Looks like a case for conditional compilation.
> 
> Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
> by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
> only needed for performance study of the algorithms, which likely won't be done often.
> 
> >
> > Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> > a setting in m3makefile?
> >
> > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
> >
> >       Mika
> >
> >
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140110/fe64b4e8/attachment.html>

From mika at async.caltech.edu  Fri Jan 10 16:27:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 10 Jan 2014 07:27:10 -0800
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
	<COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
Message-ID: <20140110152711.036F21A208F@async.async.caltech.edu>


>Mika=2C how did you find this?Control-c every so often and look at callstac=
>k?Or a lot of stepping?

"Poor man's profiling" -- since profiling is usually broken in Modula-3.  It worked back with SRC M3 but that was a while ago :-)

1. run program in gdb
2. close eyes
3. count to three
4. hit ctrl-C
5. open eyes

(repeat a few times)

"close eyes" is important!


From dragisha at m3w.org  Sun Jan 12 09:49:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 09:49:23 +0100
Subject: [M3devel] Conversion to Git
Message-ID: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>

Hi,

I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.

For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.

alexb
darko
dbenavid
dragisha
eiserlohpp
hosking
hudson
jkrell
khaeusler
kschleiser
m3
mand
micha
mika
mm
neels
pmckinna
rcoleburn
rforb
rillig
rodney
sk
stsp
thielema
uamoore
wagner
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment.sig>

From mika at async.caltech.edu  Sun Jan 12 18:53:17 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:53:17 -0800
Subject: [M3devel] CM3 crashing
Message-ID: <20140112175317.9F0031A2080@async.async.caltech.edu>

Hi m3devel,

Anyone have an idea what's going on here?

I have program that compiles with PM3 (FreeBSD4):

...
new source -> compiling ../src/Main.m3
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

new opaque info -> recompiling HParseStd.m3
...

With CM3 at head (AMD64_LINUX (userthreads)), the following happens:

--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure


***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
***

Abort

(gdb) run -x
Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

Program received signal SIGSEGV, Segmentation fault.
0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
36            u.get_info (RInfo);  
(gdb) where
#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading variable>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variable>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:1289
#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading variable>) at ../src/misc/CG.m3:2547
#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable>) at ../src/values/Procedure.m3:358
#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) at ../src/values/Value.m3:61
#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variable>) at ../src/exprs/ProcExpr.m3:96
#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:145
#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading variable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error reading variable>) at ../src/values/Formal.m3:550
#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading variable>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:459
#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:131
#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:431
#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:91
#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading variable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker.m3:540
#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading variable>) at ../src/stmts/ReturnStmt.m3:55
#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:345
#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable>) at ../src/stmts/IfStmt.m3:105
#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:339
#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading variable>) at ../src/stmts/BlockStmt.m3:103
#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading variable>) at ../src/values/Procedure.m3:577
#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading variable>) at ../src/values/Procedure.m3:541
#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading variable>) at ../src/misc/ProcBody.m3:170
#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error reading variable>) at ../src/misc/ProcBody.m3:69
#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:905
#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:837
#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading variable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error reading variable>) at ../src/misc/M3Front.m3:66
#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variable>) at ../src/Builder.m3:1705
#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error reading variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m3:1025
#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_AicXUJ_shared=<error reading variable>, 
    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading variable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variable>) at ../src/QMachine.m3:1773
#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_file=<error reading variable>) at ../src/M3Build.m3:679
#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading variable>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../src/Main.m3:231
#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe390) at _m3main.c:22
(gdb) 

Looks like something very basic...





From mika at async.caltech.edu  Sun Jan 12 18:58:30 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:58:30 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175317.9F0031A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
Message-ID: <20140112175830.C51881A2080@async.async.caltech.edu>

Sigh...

    188 PROCEDURE PutErr (u: U;  a, b, c: TEXT := NIL) =
    189   BEGIN
    190     u.note_error ("********* M3CG_Check ERROR *********** " & a & b & c);
    191     u.child.comment ("********* M3CG_Check ERROR *********** ", a, b, c);
    192     INC (u.n_errors);
    193   END PutErr;

Ah.... ok....

"../src/Main.m3", line 129: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 131: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 133: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"

The code is:

              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
            |
              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
            |
              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
 
I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 

mika writes:
>Hi m3devel,
>
>Anyone have an idea what's going on here?
>
>I have program that compiles with PM3 (FreeBSD4):
>
>...
>new source -> compiling ../src/Main.m3
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>new opaque info -> recompiling HParseStd.m3
>...
>
>With CM3 at head (AMD64_LINUX (userthreads)), the following happens:
>
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>
>***
>*** runtime error:
>***    Attempt to reference an illegal memory location.
>***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
>***
>
>Abort
>
>(gdb) run -x
>Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
>[Thread debugging using libthread_db enabled]
>Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>Program received signal SIGSEGV, Segmentation fault.
>0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M
>3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>36            u.get_info (RInfo);  
>(gdb) where
>#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable
>>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading varia
>ble>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variabl
>e>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
>#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading
> variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
>#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error re
>ading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3
>:1289
>#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading varia
>ble>) at ../src/misc/CG.m3:2547
>#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable
>>) at ../src/values/Procedure.m3:358
>#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) a
>t ../src/values/Value.m3:61
>#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variab
>le>) at ../src/exprs/ProcExpr.m3:96
>#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>)
> at ../src/exprs/Expr.m3:145
>#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error readi
>ng variable>) at ../src/values/Formal.m3:550
>#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading varia
>ble>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error readi
>ng variable>) at ../src/values/Formal.m3:459
>#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:131
>#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:
>431
>#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:91
>#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading va
>riable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker
>.m3:540
>#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading vari
>able>) at ../src/stmts/ReturnStmt.m3:55
>#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:345
>#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable
>>) at ../src/stmts/IfStmt.m3:105
>#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:339
>#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading varia
>ble>) at ../src/stmts/BlockStmt.m3:103
>#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading varia
>ble>) at ../src/values/Procedure.m3:577
>#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading vari
>able>) at ../src/values/Procedure.m3:541
>#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading varia
>ble>) at ../src/misc/ProcBody.m3:170
>#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error readin
>g variable>) at ../src/misc/ProcBody.m3:69
>#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading va
>riable>) at ../src/values/Module.m3:905
>#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable
>>) at ../src/values/Module.m3:837
>#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
>#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading var
>iable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error readi
>ng variable>) at ../src/misc/M3Front.m3:66
>#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>
>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variab
>le>) at ../src/Builder.m3:1705
>#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
>#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
>#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading vari
>able>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
>#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error readi
>ng variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m
>3:1025
>#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading
> variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<erro
>r reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
>    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
>#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading var
>iable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error re
>ading variable>, M3_AicXUJ_shared=<error reading variable>, 
>    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
>#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading va
>riable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
>#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading varia
>ble>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
>#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variab
>le>) at ../src/QMachine.m3:1773
>#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
>#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd
>56fi_file=<error reading variable>) at ../src/M3Build.m3:679
>#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading va
>riable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
>#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading varia
>ble>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
>#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3
>_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
>#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, 
>M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
>#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
>#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at
> ../src/Main.m3:231
>#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading va
>riable>) at ../src/runtime/common/RTLinker.m3:408
>#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading varia
>ble>) at ../src/runtime/common/RTLinker.m3:115
>#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variab
>le>) at ../src/runtime/common/RTLinker.m3:124
>#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe3
>90) at _m3main.c:22
>(gdb) 
>
>Looks like something very basic...
>
>


From hosking at cs.purdue.edu  Sun Jan 12 21:30:34 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 12 Jan 2014 15:30:34 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175830.C51881A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
Message-ID: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>

Are you saying passing these used to work in PM3?
Sounds like a front-end bug.  I?m curious what changed to break it.

On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

> 
> The code is:
> 
>              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
>            |
>              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
>            |
>              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
> 
> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484







From dragisha at m3w.org  Sun Jan 12 22:45:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 22:45:32 +0100
Subject: [M3devel] Conversion to Git
In-Reply-To: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
References: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
Message-ID: <E4D3225F-F1C5-4660-9EFB-6EA49218733F@m3w.org>

Thanks to everybody who send me infomation, esp. to Peter Eiserloh who gave me information + idea how to fill rest. Missing are emails from some Elego people and I expect I will have that soon.

I will keep you all posted on conversion status. Current version is on Github (dragisha/cm3), and I am checking how it matches with local CVS checkouts.

On 12 Jan 2014, at 09:49, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Hi,
> 
> I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.
> 
> For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.
> 
> alexb
> darko
> dbenavid
> dragisha
> eiserlohpp
> hosking
> hudson
> jkrell
> khaeusler
> kschleiser
> m3
> mand
> micha
> mika
> mm
> neels
> pmckinna
> rcoleburn
> rforb
> rillig
> rodney
> sk
> stsp
> thielema
> uamoore
> wagner
> --
> Dragi?a Duri?
> dragisha at m3w.org
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment.sig>

From mika at async.caltech.edu  Mon Jan 13 01:25:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 16:25:39 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
Message-ID: <20140113002539.EC4EE1A2080@async.async.caltech.edu>

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>


From hosking at cs.purdue.edu  Mon Jan 13 16:46:36 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 10:46:36 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <09749A02-B9B9-4C74-8CB4-73379E91385B@cs.purdue.edu>


On Jan 12, 2014, at 7:25 PM, mika at async.caltech.edu wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

Interesting.  Needs exploration.  This should be filed as a bug.  The workaround is to wrap the builtin calls (Word.Xor, etc.) in a local procedure so that you have an addressable procedure.

> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?

Yes, they are inlined, but there *is* an implementation in Word.m3 (generated from GenWord.mg).

> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?

It must be the threads implementation, since the same collector is used with both pthreads and user threads, and I believe that user threads don?t suffer from this problem.  My intuition is that it may be a problem with getting stack roots on particular platforms, probably because the stack bounds are not being computed correctly for a given architecture, or perhaps because reference alignment is not being observed correctly.  For example, is it possible on x86_64 to have references stored in the stack aligned on 32-bit instead of 64-bit boundaries?

I notice that RTThread.i3 says the pointer alignment is BYTESIZE(ADDRESS) on all platforms.  Once upon a time the alignment was platform-dependent, so I wonder if we lost something somewhere with all the backend and porting work.  It may be that the backend needs to be instructed to align all references appropriately in the stacks.

One way to test all this might be to see if the thread stress tester causes things to break on older 32-bit platforms, where I expect all pointers will be aligned properly.

Also, I?ve not inspected the code in ThreadPThreadC.c recently (it?s changed since I last looked), but I wonder if the stack bounds are being computed correctly so that the collector can find all the stack roots.

I?d really like to get to the bottom of this problem.

> Tony Hosking writes:
>> Are you saying passing these used to work in PM3?
>> Sounds like a front-end bug.  I=92m curious what changed to break it.
>> 
>> On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> The code is:
>>> =20
>>>             HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Xor), ab)
>>>           |
>>>             HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Or), ab)
>>>           |
>>>             HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.And), ab)
>>> =20
>>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>> 
>> 
>> 
>> Antony Hosking | Associate Professor | Computer Science | Purdue =
>> University
>> 305 N. University Street | West Lafayette | IN 47907 | USA
>> Mobile +1 765 427 5484
>> 
>> 
>> 
>> 



From hosking at cs.purdue.edu  Mon Jan 13 17:03:38 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 11:03:38 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <EA2F5D60-CBC8-47A1-BC34-B3981DAFE4BD@cs.purdue.edu>

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:

> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 
> 
> Regards Peter
> 
> 
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/202c6d98/attachment.html>

From rcolebur at SCIRES.COM  Mon Jan 13 20:16:41 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:16:41 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy

Randy C. Coleburn, CISSP-ISSEP, GCED
Corporate Information Security Officer (CISO)
Scientific Research Corporation
2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339
voice: (770) 989-9464, email: RColeburn at SciRes.com<mailto:RColeburn at SciRes.com>, fax: (770) 989-9497

[logo_SRC]
"Technology Driven-Customer Focused"

Quality Policy:  "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements."

CONFIDENTIALITY NOTICE:  This email constitutes an electronic communication within the meaning of the Electronic Communications Privacy Act, 18 U.S.C. 2510, and its disclosure is strictly limited to the named recipient(s) intended by the sender of this message.  This email, and any attachments, may contain confidential and/or proprietary information of Scientific Research Corporation.  If you are not a named recipient, any copying, using, disclosing or distributing to others the information in this email and attachments is STRICTLY PROHIBITED.  If you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts or hard copies of the email and attachments.

EXPORT COMPLIANCE NOTICE:  This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR).  Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer.  In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization.  By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:


Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 11339 bytes
Desc: image002.jpg
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment.jpg>

From rcolebur at SCIRES.COM  Mon Jan 13 20:55:44 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:55:44 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/3dbd395c/attachment.html>

From hosking at cs.purdue.edu  Mon Jan 13 21:02:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 15:02:57 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <227E8C61-B070-49FF-8AD5-E23420E9C1B0@cs.purdue.edu>

Is this from the head of source?
Also, please run with @M3paranoidgc flag.


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 
>  
> C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386
>  
> C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2013-12-17 17:28:52
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> Here is the output:
>  
> C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> This last assert repeats indefinitely until you press CTRL-C.
>  
> --Randy
>  
>  
> From: Coleburn, Randy 
> Sent: Monday, January 13, 2014 2:17 PM
> To: Tony Hosking; Peter McKinna
> Cc: m3devel
> Subject: Re: [M3devel] CM3 crashing
>  
> Tony et al:
>  
> The thread test program fails for me on 64-bit Windows 7.
>  
> I?ve listed output from a couple of runs below:
>  
> C:\cm3\Sandbox>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2014-01-07 06:55:14
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> FIRST RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\runtime\common\RTCollector.m3", line 418
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
>                ?this last assertion repeats until you press CTRL-C to abort the program?
>  
> SECOND RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
> ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
> ....
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x776f22d2
> ***
>  
> Stack trace:
>    FP         PC      Procedure
> ---------  ---------  -------------------------------
> 0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
> 0xa6f9d0  0x776f22d2  <???>
> 0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
> 0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
> 0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fb90  0x76db336a  <???>
> 0xa6fbd0  0x7770bf32  <???>
> .........  .........  ... more frames ...
>  
> Maybe this info will be useful in tracking down the problem.
>  
> --Randy
>  
>  
> From: Tony Hosking [mailto:hosking at cs.purdue.edu] 
> Sent: Monday, January 13, 2014 11:04 AM
> To: Peter McKinna
> Cc: m3devel
> Subject: EXT:Re: [M3devel] CM3 crashing
>  
> Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
> In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.
>  
> As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?
>  
> If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.
>  
> On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:
>  
> 
> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
> 
> Regards Peter
>  
>  
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/c56a7e47/attachment.html>

From rcolebur at SCIRES.COM  Mon Jan 13 21:13:55 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 20:13:55 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:


I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/6198676d/attachment.html>

From rcolebur at SCIRES.COM  Mon Jan 13 22:02:52 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 21:02:52 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/eea82060/attachment.html>

From jay.krell at cornell.edu  Tue Jan 14 00:22:12 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 13 Jan 2014 23:22:12 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>,
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W762DFE2FF1E39C9147F90EE6BC0@phx.gbl>

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: 
 
  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html   
 
 
 And disappointed. Cooperative suspend will fix this someday. 
 
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: hosking at cs.purdue.edu
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.



 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the
 finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:



 





Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much
 always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok
 to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the
 alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.



It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 





 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/45717d65/attachment.html>

From rcolebur at SCIRES.COM  Tue Jan 14 01:11:07 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Tue, 14 Jan 2014 00:11:07 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?

Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Monday, January 13, 2014 6:22 PM
To: Coleburn, Randy; Tony
Cc: m3devel
Subject: EXT:RE: [M3devel] CM3 crashing

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html


 And disappointed. Cooperative suspend will fix this someday.


 - Jay

________________________________
From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
To: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
Subject: Re: [M3devel] CM3 crashing
Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/7dd888c2/attachment.html>

From jay.krell at cornell.edu  Tue Jan 14 01:21:49 2014
From: jay.krell at cornell.edu (Jay K)
Date: Tue, 14 Jan 2014 00:21:49 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W14788FF4A8D081652DA79DE6BF0@phx.gbl>

I don't know. We can try writing a stress test for it. It looks like one is available.
Agreed -- work out the bugs in the native Windows and Posix systems.
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: jay.krell at cornell.edu; hosking at cs.purdue.edu
Date: Tue, 14 Jan 2014 00:11:07 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Jay:
 
The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?
 
Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Monday, January 13, 2014 6:22 PM

To: Coleburn, Randy; Tony

Cc: m3devel

Subject: EXT:RE: [M3devel] CM3 crashing


 

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:


 

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html  


 

 

 And disappointed. Cooperative suspend will fix this someday. 

 

 

 - Jay

 






From: 
rcolebur at SCIRES.COM

To: hosking at cs.purdue.edu

Date: Mon, 13 Jan 2014 21:02:52 +0000

CC: m3devel at elegosoft.com

Subject: Re: [M3devel] CM3 crashing

Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll
 commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.


 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive)
 to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user
 threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without
 keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:


 




Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it
 with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with
 code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the
 read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only
 way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.


It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 




 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 




 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/2693506b/attachment.html>

From peter.mckinna at gmail.com  Mon Jan 13 04:54:08 2014
From: peter.mckinna at gmail.com (Peter McKinna)
Date: Mon, 13 Jan 2014 14:54:08 +1100
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>

Hey,

  I was trying to get a handle on that problem last year. The threadtest
program is really a stress tester of the collector/allocator with pthreads.
If you run it with just the tests read and alloc you pretty much always get
a crash. If you run them with paranoidgc it will crash in the heap checker.
Tony thought it was a clear cut  problem of the roots of some ref not being
found on a thread stack. I mucked around with code to get some output and
the stacks looked ok to me but I could be wrong. All stacks are checked
whilst the threads are blocked in a signal handler, the design of which
looks fine as far as I can tell. This test is characterised by some slow
threads (the read threads) and a bunch of fast threads (the alloc threads).
Even if you modify the test to have only one read thread the problem
occurs. I have had misgivings about mixing signals and threads having been
bitten many years ago, but really this is the only way the collector can
get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the
collector itself. One thing I noticed is that in
ThreadPThread__ProcessStopped  the second call to   p(context, ((char
*)context) + sizeof(ucontext_t)); according to the comment is to process
the registers. But the registers should already be on the stack and anyway
this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the
collector has always been at the heart of m3 programs.

Regards Peter



On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a
> reason to switch to CM3).
>
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
>
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
>
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/9bfaddc0/attachment.html>

From jay.krell at cornell.edu  Wed Jan 15 06:01:16 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 05:01:16 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>,
	<20140112175830.C51881A2080@async.async.caltech.edu>,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>

Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/7fcd3a4f/attachment.html>

From jay.krell at cornell.edu  Wed Jan 15 07:48:30 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 06:48:30 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>, ,
	<20140112175830.C51881A2080@async.async.caltech.edu>, ,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>, ,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>, ,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>,
	<COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
Message-ID: <COL130-W18EA0744A6E2E144FEB51AE6BE0@phx.gbl>

ok, I augmented the test a bit and commited it to CVS and I can see it fail now.
Does the test make sense to people?
 
Thanks,
 - Jay

 
From: jay.krell at cornell.edu
To: peter.mckinna at gmail.com; mika at async.caltech.edu
Date: Wed, 15 Jan 2014 05:01:16 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing




Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/cb7c9be3/attachment.html>

From jay.krell at cornell.edu  Sat Jan 18 07:22:53 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 18 Jan 2014 06:22:53 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <COL130-W549E4FE51DBF493AF61101E6BB0@phx.gbl>

 This program also doesn't behave as expected, native, nothing to do with wow64. 
 Anyone else please confirm: 
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it  
   3) their not being met -- stuff gets printed 
 This is in the CVS repository, scratch/wow64stack/sync2.cpp 
 I am following up further. 
 Maybe we should get cooperative suspend really going? 
Thank you.
 - Jay 
 
#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value; 
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/26ed6280/attachment.html>

From rcolebur at SCIRES.COM  Sat Jan 18 08:11:45 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Sat, 18 Jan 2014 07:11:45 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

I'm very concerned about the threading not working properly on both 32-bit and 64-bit Windows.

The Thread Test program crashes for me on both platforms.

I haven't tried your new test program yet.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Saturday, January 18, 2014 1:23 AM
To: m3devel
Subject: EXT:[M3devel] Win32 SuspendThread?

 This program also doesn't behave as expected, native, nothing to do with wow64.
 Anyone else please confirm:
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it
   3) their not being met -- stuff gets printed
 This is in the CVS repository, scratch/wow64stack/sync2.cpp
 I am following up further.
 Maybe we should get cooperative suspend really going?
Thank you.
 - Jay

#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value;
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/ee809938/attachment.html>

From dragisha at m3w.org  Sat Jan 18 09:24:11 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 18 Jan 2014 09:24:11 +0100
Subject: [M3devel] Cooperative suspend?
Message-ID: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>

Jay,

What exactly do you mean? Isn?t it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?

dd
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment.sig>

From mika at async.caltech.edu  Sun Jan 19 04:40:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 18 Jan 2014 19:40:06 -0800
Subject: [M3devel] how is this possible?
Message-ID: <20140119034006.34F511A208F@async.async.caltech.edu>


Any hypothesis is welcome...

(311)truffles:~/t/btc/hparse/src>../AMD64_LINUX/hparse and.scm 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../AMD64_LINUX/BDDSchemeStubs.m3", line 619
***

Abort
(312)truffles:~/t/btc/hparse/src>

    613 PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } =
    614   BEGIN
    615     IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END;
    616     IF NOT ISTYPE(x,BDD.T) THEN
    617       RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
    618     END;
    619     RETURN x
    620   END ToModula_BDD_T;


The object is of type BDD.Root (I checked the TYPECODE), which is
declared thus:


TYPE
  Root = T BRANDED Brand & " Root" OBJECT
    mu : MUTEX; (* as yet unused *)
    id : CARDINAL;
    tab : BDDTripleHash.T;
    cache : ARRAY Op OF BDDTripleHash.T;
  END;

SchemeObject.T is the same as REFANY

I'm not even sure where to start debugging this.  The code "should work"...

Indeed:

> (rttype-typecode a)
144
> (RTBrand.GetByTC 144)
"BDD 0.1 Root"
> (RTBrand.GetByTC 143)
"BDD 0.1"
> (RTType.Supertype 143)
4

(4 is REFANY)

> (BDD.And a a)
EXCEPTION! RuntimeError! An explicit or implicit NARROW() operation failed.
> a-and-a
<Modula-3 object : BDDImpl.Root, BRANDED BDD 0.1 Root>
> (rttype-typecode a-and-a)
144


     Mika




From jay.krell at cornell.edu  Sun Jan 19 02:02:56 2014
From: jay.krell at cornell.edu (Jay)
Date: Sat, 18 Jan 2014 17:02:56 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>

For native I'm less worried now. For wow64 I'm still worried & hope to follow up further.

 - Jay

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:

> Jay:
>  
> I?m very concerned about the threading not working properly on both 32-bit and 64-bit Windows.
>  
> The Thread Test program crashes for me on both platforms.
>  
> I haven?t tried your new test program yet.
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Saturday, January 18, 2014 1:23 AM
> To: m3devel
> Subject: EXT:[M3devel] Win32 SuspendThread?
>  
>  This program also doesn't behave as expected, native, nothing to do with wow64. 
>  Anyone else please confirm: 
>    1) my expectations -- it should never print anything
>    2) their importance  -- garbage collector depends on it  
>    3) their not being met -- stuff gets printed 
>  This is in the CVS repository, scratch/wow64stack/sync2.cpp 
>  I am following up further. 
>  Maybe we should get cooperative suspend really going? 
> Thank you.
>  - Jay 
>  
> #include <stdio.h>
> #include <windows.h>
> volatile long value;
> unsigned long __stdcall Thread(PVOID parameter)
> {
>   while (1)
>    InterlockedIncrement(&value);
>   return 0;
> }
> int __cdecl main()
> {
>   HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
>   UINT i = 0;
>   while (1)
>   {
>     i += 1;
>     if (SuspendThread(thread) == (DWORD)-1)
>     {
>       printf("suspend failed %X\n", GetLastError());
>       Sleep(1);
>       continue;
>     }
>     volatile long a = value; 
>     volatile long b = value;
>     if (a != b)
>     {
>      printf("%d %d %d %d\n", i, a, b, b - a);
>     }
>     ResumeThread(thread);
>   }
> }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/21485aaa/attachment.html>

From mika at async.caltech.edu  Sun Jan 19 10:41:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 01:41:39 -0800
Subject: [M3devel] something wrong with ISTYPE ...
Message-ID: <20140119094139.3917E1A208F@async.async.caltech.edu>


Hi m3devel,

There is definitely something wrong with ISTYPE.  The problem I reported earlier goes away if I change the code like so:

(416)truffles:~/t/btc/hparse/AMD64_LINUX>diff -c BDDSchemeStubs.m3.old BDDSchemeStubs.m3
*** BDDSchemeStubs.m3.old       2014-01-19 01:13:33.000000000 -0800
--- BDDSchemeStubs.m3   2014-01-19 01:36:52.000000000 -0800
***************
*** 613,622 ****
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     IF NOT ISTYPE(x,BDD.T) THEN
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END;
!     RETURN x
    END ToModula_BDD_T;
  
  
--- 613,624 ----
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     TYPECASE x OF
!       BDD.T(xx) => RETURN xx
!     ELSE
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END
! 
    END ToModula_BDD_T;

This shouldn't be possible... it's the same object as before, with the
same return type.  I don't get an exception (I shouldn't either)---the
return just behaves normally and the program works.  Why doesn't it work
with ISTYPE but only with TYPECASE?

BTW, this is an old bug.  Same behavior with ancient PM3.  Where to look
in the code?

    Mika


From adacore at marino.st  Sun Jan 19 11:03:27 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 11:03:27 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Message-ID: <52DBA2EF.4020101@marino.st>

Hi all,

I've been trying to port cm3 to DragonFly x86-64.
I'm able to build cm3cg on DragonFly, and use it to build libraries
(e.g. libm3core) that were targeted for AMD64_FREEBSD.

However, when I set the HOST to AMD64_FREEBSD and TARGET to
AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
target machine: TARGET = AMD64_DRAGONFLY" (context below)

=========================================================
> /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> ROOT        = /work/lang/modula3/work/modula3-5.8.6
> CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> M3GDB       = yes
> M3OSTYPE    = POSIX
> TARGET      = AMD64_DRAGONFLY
> GCC_BACKEND = yes
> INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> GREP        = egrep
> GMAKE       = gmake
> TMPDIR      = /var/tmp
> EXE         = 
> SL          = /
> TAR         = tar
> CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> CM3VERSION  = 5.8.6
> CM3VERSIONNUM = 050806
> CM3LASTCHANGED = 2010-04-11
> FIND = /usr/bin/find
> EGREP = egrep
> === package m3-libs/m3core ===
>  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> --- building in AMD64_DRAGONFLY ---
> 
> ignoring ../src/m3overrides
> 
> 
> Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> 
>  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> *** [assembler-code] Error code 1

=========================================================

I thought I added all the target definitions but apparently I've missed
something.  I also realize the current repo looks a lot different, but I
wanted to use the last official release.

I've attached a concatenated file of patches showing what I've changed.
 Could somebody suggest what data is missing that causes the error above?

Regards,
John
-------------- next part --------------
--- m3-libs/libm3/src/os/POSIX/m3makefile.orig	2009-07-24 10:02:40.000000000 +0000
+++ m3-libs/libm3/src/os/POSIX/m3makefile
@@ -84,6 +84,7 @@ local readonly DefaultOSName = {
    "MIPS64_OPENBSD" : "OpenBSD",
    "SPARC64_SOLARIS" : "Solaris",
    "I386_OPENBSD" : "OpenBSD",
+   "AMD64_DRAGONFLY" : "DragonFly",
    "AMD64_FREEBSD" : "FreeBSD",
    "PA32_HPUX" : "HPUX",
    "PA64_HPUX" : "HPUX",
@@ -92,6 +93,7 @@ local readonly DefaultOSName = {
    "AIX" : "AIX",
    "CYGWIN" : "Cygwin",
    "DARWIN" : "Darwin",
+   "DRAGONFLY" : "DragonFly",
    "FREEBSD" : "FreeBSD",
    "HPUX" : "HPUX",
    "INTERIX" : "Interix",
@@ -156,6 +158,7 @@ local readonly DefaultArch = {
    "SPARC64_SOLARIS" : "sparc64",
    "I386_OPENBSD" : "i686",
    "AMD64_FREEBSD" : "AMD64",
+   "AMD64_DRAGONFLY" : "AMD64",
    "PA32_HPUX" : "hppa",
    "PA64_HPUX" : "hppa64",
 
--- m3-libs/m3core/src/Csupport/m3makefile.orig	2009-07-24 10:51:01.000000000 +0000
+++ m3-libs/m3core/src/Csupport/m3makefile
@@ -11,6 +11,7 @@ readonly _CSupportPieces = {
 % "ALPHA_OSF"  : [ "little-endian" ],
 % "AIX386"     : [ "little-endian" ],
   "AMD64_DARWIN": [ "little-endian" ],
+  "AMD64_DRAGONFLY": [ "little-endian" ],
   "AMD64_FREEBSD": [ "little-endian" ],
   "AMD64_OPENBSD": [ "little-endian" ],
   "AMD64_NETBSD": [ "little-endian" ],
--- m3-libs/m3core/src/float/m3makefile.orig	2009-07-24 10:51:24.000000000 +0000
+++ m3-libs/m3core/src/float/m3makefile
@@ -13,6 +13,7 @@ _FloatPieces = {
 % "ALPHA_OSF"  : _float_le,
 % "AIX386"     : _float_le,
   "AMD64_DARWIN": _float_le,
+  "AMD64_DRAGONFLY": _float_le,
   "AMD64_FREEBSD": _float_le,
   "AMD64_OPENBSD": _float_le,
   "AMD64_NETBSD": _float_le,
--- m3-libs/m3core/src/m3core.h.orig	2010-03-28 10:07:45.000000000 +0000
+++ m3-libs/m3core/src/m3core.h
@@ -207,7 +207,8 @@ typedef INTEGER m3_uid_t;
 #define PTHREAD_TO_M3(x)   ((m3_pthread_t)(size_t)(x))
 #define PTHREAD_FROM_M3(x) ((pthread_t)(size_t)(x))
 
-#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__)
+#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__OpenBSD__) || defined(__DragonFly__)
 #define HAS_STAT_FLAGS
 #endif
 
--- m3-libs/m3core/src/platforms.quake.orig	2010-03-28 10:09:07.000000000 +0000
+++ m3-libs/m3core/src/platforms.quake
@@ -1,5 +1,6 @@
 _TARGET_OS = {
   "AMD64_DARWIN"  : "DARWIN",
+  "AMD64_DRAGONFLY" : "DRAGONFLY",
   "AMD64_FREEBSD" : "FREEBSD",
   "AMD64_LINUX"   : "LINUX",
   "AMD64_NETBSD"  : "NETBSD",
@@ -32,6 +33,7 @@ _TARGET_ENDIAN = {
 % This is not computed based on architecture
 % because some architectures vary, e.g. PPC_NT is little endian, PPC_DARWIN is big.
   "AMD64_DARWIN"    : "LITTLE",
+  "AMD64_DRAGONFLY" : "LITTLE",
   "AMD64_FREEBSD"   : "LITTLE",
   "AMD64_LINUX"     : "LITTLE",
   "AMD64_NETBSD"    : "LITTLE",
--- m3-libs/m3core/src/runtime/POSIX/RTSignalC.c.orig	2010-04-16 12:20:56.000000000 +0000
+++ m3-libs/m3core/src/runtime/POSIX/RTSignalC.c
@@ -167,7 +167,7 @@ static size_t GetPC(void* xcontext)
 #elif defined(__NetBSD__)
     _UC_MACHINE_PC(context)
 
-#elif defined(__FreeBSD__)
+#elif defined(__FreeBSD__) || defined(__DragonFly__)
 #if defined(__amd64)
     context->uc_mcontext.mc_rip
 #elif defined(__i386)
--- m3-libs/m3core/src/runtime/common/Compiler.tmpl.orig	2009-07-13 09:09:35.000000000 +0000
+++ m3-libs/m3core/src/runtime/common/Compiler.tmpl
@@ -21,7 +21,7 @@ readonly Compiler_i3 = [
   "               SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD,",
   "               SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD,",
   "               PA32_HPUX, PA64_HPUX, ARM_DARWIN, I386_INTERIX,",
-  "               AMD64_NETBSD, AMD64_OPENBSD };",
+  "               AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY };",
   "",
   "CONST",
   "  ThisOS       = OS." & OS_TYPE & ";",
--- m3-libs/m3core/src/thread.quake.orig	2009-12-20 11:01:02.000000000 +0000
+++ m3-libs/m3core/src/thread.quake
@@ -12,6 +12,7 @@ readonly NO_USER_THREADS =
     "AMD64_FREEBSD":1,
     "AMD64_OPENBSD":1,
     "AMD64_NETBSD":1,
+    "AMD64_DRAGONFLY":1,
     "AMD64_LINUX":1,
     "ARM_LINUX":1,
     "MIPS32_IRIX":1,
--- m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c.orig	2009-12-21 05:35:34.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c
@@ -2,7 +2,7 @@
 /* All rights reserved.                                            */
 /* See the file COPYRIGHT-PURDUE for a full description.           */
 
-#ifndef __FreeBSD__
+#if !defined(__FreeBSD__) && !defined(__DragonFly__)
 
 /* avoid empty file */
 
--- m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c.orig	2009-12-12 14:48:10.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c
@@ -23,7 +23,7 @@
 #endif
 #endif
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 #define M3_DIRECT_SUSPEND
 #endif
 
@@ -177,7 +177,7 @@ void ThreadPThread__sem_post(void)      
 void ThreadPThread__sem_getvalue(void)  { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 void ThreadPThread__sigsuspend(void)    { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 
 int
 ThreadPThread__SuspendThread (m3_pthread_t mt)
--- m3-libs/m3core/src/time/POSIX/m3makefile.orig	2009-07-24 07:26:15.000000000 +0000
+++ m3-libs/m3core/src/time/POSIX/m3makefile
@@ -11,6 +11,7 @@ readonly _DateImpls = {
 % "ALPHA_OSF"  : "DateBsd",
 % "AIX386"     : "DateBsd",
   "AMD64_DARWIN": "DateBsd",
+  "AMD64_DRAGONFLY": "DateBsd",
   "AMD64_FREEBSD": "DateBsd",
   "AMD64_OPENBSD": "DateBsd",
   "AMD64_NETBSD": "DateBsd",
@@ -63,6 +64,7 @@ readonly _DateImpls = {
 % "AIX"     : "DateBsd",
   "CYGWIN"  : "DatePosix",
   "DARWIN"  : "DateBsd",
+  "DRAGONFLY" : "DataBsd",
   "FREEBSD" : "DateBsd",
   "HPUX"    : "DatePosix",
   "INTERIX" : "DatePosix",
--- m3-libs/m3core/src/unix/Common/UtimeC.c.orig	2009-12-11 11:18:34.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/UtimeC.c
@@ -56,7 +56,8 @@ struct timeval
 #endif
 }
 
-#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__APPLE__)
+#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__APPLE__) || defined(__DragonFly__)
 #define M3BSD
 #endif
 
--- m3-libs/m3core/src/unix/Common/Uucontext.c.orig	2009-06-29 19:20:40.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/Uucontext.c
@@ -29,6 +29,7 @@ see http://www.opengroup.org/onlinepubs/
     || defined(__linux) \
     || defined(__NetBSD__) \
     || defined(__hpux) \
+    || defined(__DragonFly__) \
     || defined(__FreeBSD__) \
 
 int Uucontext__getcontext(ucontext_t* context)
--- m3-libs/m3core/src/unix/Common/m3makefile.orig	2009-07-19 00:28:17.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/m3makefile
@@ -81,7 +81,7 @@ if equal (TARGET, "MIPS64_OPENBSD") or e
         or equal (TARGET, "PA32_HPUX") or equal (TARGET, "PA64_HPUX")
         or equal (TARGET, "NT386") or equal (TARGET, "AMD64_FREEBSD")
         or equal (TARGET, "AMD64_OPENBSD") or equal (TARGET, "AMD64_NETBSD")
-        or equal (TARGET, "I386_INTERIX")
+        or equal (TARGET, "I386_INTERIX") or equal (TARGET, "AMD64_DRAGONFLY")
 
     Interface("Uucontext")
 
--- m3-libs/m3core/src/unix/m3makefile.orig	2009-07-28 18:49:13.000000000 +0000
+++ m3-libs/m3core/src/unix/m3makefile
@@ -6,6 +6,7 @@ readonly _UnixPieces = {
 % "AIX386"          : [ "aix-ps2-1-2",                             "uin-common" ],
 % "ALPHA_OSF"       : [ "osf-1.generic", "osf-1.ALPHA_OSF",        "uin-common" ],
   "AMD64_DARWIN"    : [ "darwin-common", "darwin-amd64",           "uin-len" ],
+  "AMD64_DRAGONFLY" : [ "freebsd-common",                          "uin-len" ],
   "AMD64_FREEBSD"   : [ "freebsd-common",                          "uin-len" ],
   "AMD64_LINUX"     : [ "linux-common",                            "uin-common" ],
   "AMD64_NETBSD"    : [ "netbsd-common",                           "uin-len" ],
--- m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY.orig	2014-01-16 23:14:23.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY
@@ -0,0 +1,9 @@
+readonly TARGET = "AMD64_DRAGONFLY"
+readonly GNU_PLATFORM = "amd64-dragonfly" % "cpu-os" string for GNU
+
+m3back_flags = "-gstabs+ -m64 -fPIC -funwind-tables"
+SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC"
+readonly SYSTEM_ASM = "as -64"
+
+include("AMD64.common")
+include("DragonFly.common")
--- m3-sys/cminstall/src/config-no-install/DragonFly.common.orig	2014-01-16 23:18:36.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/DragonFly.common
@@ -0,0 +1,24 @@
+readonly TARGET_OS = "DRAGONFLY"
+
+GNU_MAKE = "gmake"
+
+include("Unix.common")
+
+SYSTEM_LIBS{"ODBC"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lodbc" ]
+SYSTEM_LIBS{"POSTGRES95"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lpq" ]
+SYSTEM_LIBS{"X11"} = ["-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ]
+
+DRAGONFLY_CC_APPEND = " -z origin"
+                    & " -z now"
+                    & " -Bsymbolic"
+
+DRAGONFLY_LD_APPEND = DRAGONFLY_CC_APPEND
+                    & " -Wl,--fatal-warnings"
+                    & " -Wl,--warn-common"
+                    & " -Wl,-rpath,/usr/local/lib"
+                    & " -Wl,-rpath,\\$ORIGIN/../lib "
+
+SYSTEM_LD = SYSTEM_CC & DRAGONFLY_LD_APPEND
+SYSTEM_CC = SYSTEM_CC & DRAGONFLY_CC_APPEND
+
+include("gnuld.common")
--- m3-sys/m3cc/src/boot.py.orig	2008-05-19 13:09:03.000000000 +0000
+++ m3-sys/m3cc/src/boot.py
@@ -76,7 +76,7 @@ if SearchPath("gcc"):
     " + env)
 
   Run(gmake + "all-gmp all-mpfr all-libcpp all-libdecnumber \
-    all-build-libiberty all-libiberty configure-gcc")
+    all-build-libiberty all-libiberty configure-libcpp configure-gcc")
 
   os.chdir("gcc")
 
--- m3-sys/m3cc/src/m3makefile.orig	2010-05-14 07:00:52.000000000 +0000
+++ m3-sys/m3cc/src/m3makefile
@@ -577,7 +577,7 @@ end
 if GCC42
     m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
 else
-    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
+    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber configure-libcpp | tee -a " & Log])
 end
 
 %
--- m3-sys/m3cc/src/platforms.quake.orig	2009-07-29 17:11:08.000000000 +0000
+++ m3-sys/m3cc/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "i686-apple-darwin8",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd7",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3gdb/src/platforms.quake.orig	2010-04-21 17:06:38.000000000 +0000
+++ m3-sys/m3gdb/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "amd64-apple-darwin8.7.1",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3middle/src/Target.i3.orig	2010-02-01 14:05:35.000000000 +0000
+++ m3-sys/m3middle/src/Target.i3
@@ -33,7 +33,7 @@ TYPE
     AMD64_DARWIN, AMD64_LINUX, SPARC32_LINUX, SPARC64_LINUX,
     SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD, SPARC64_SOLARIS,
     I386_OPENBSD, AMD64_FREEBSD, PA32_HPUX, PA64_HPUX, ARM_DARWIN,
-    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, Undefined
+    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY, Undefined
   };
 
 CONST
@@ -89,7 +89,8 @@ CONST
     (* 48 *) "ARM_DARWIN",
     (* 49 *) "I386_INTERIX",
     (* 50 *) "AMD64_NETBSD",
-    (* 51 *) "AMD64_OPENBSD"
+    (* 51 *) "AMD64_OPENBSD",
+    (* 52 *) "AMD64_DRAGONFLY"
   };
 
 CONST
--- m3-sys/m3middle/src/Target.m3.orig	2010-02-01 15:06:05.000000000 +0000
+++ m3-sys/m3middle/src/Target.m3
@@ -307,6 +307,7 @@ PROCEDURE Init (system: TEXT; in_OS_name
 
     |  Systems.AMD64_NETBSD,
        Systems.AMD64_OPENBSD,
+       Systems.AMD64_DRAGONFLY,
        Systems.AMD64_FREEBSD =>
                  Jumpbuf_size              := 16_60 * Char.size;
 
--- m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c.orig	2009-04-12 10:31:23.000000000 +0000
+++ m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c
@@ -1,11 +1,11 @@
-#ifdef __FreeBSD__
+#if defined __FreeBSD__) || defined(__DragonFly__)
 #include <sys/types.h>
 #include <libutil.h>
 #endif
 
 void m3_setproctitle(const char *title)
 {
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
     setproctitle("%s", title);
 #endif
 }
--- scripts/boot-cm3-makefile-lib.tmpl.orig	2005-04-07 22:35:07.000000000 +0000
+++ scripts/boot-cm3-makefile-lib.tmpl
@@ -20,9 +20,10 @@ FILES_C  := $(wildcard *.c)
 FILES_S  := $(wildcard *.s)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C) $(FILES_S))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 
--- scripts/boot-cm3-makefile-prog.tmpl.orig	2003-07-10 22:16:05.000000000 +0000
+++ scripts/boot-cm3-makefile-prog.tmpl
@@ -19,9 +19,10 @@ FILES_MS := $(wildcard *.ms)
 FILES_C  := $(wildcard *.c)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 LIBFILES := $(shell for f in $(LIBS); do if [ -f "$${f}" ] ; then  echo $${f}; fi; done)
--- scripts/pack-crossbuild.sh.orig	2010-02-25 14:02:18.000000000 +0000
+++ scripts/pack-crossbuild.sh
@@ -23,7 +23,7 @@ if [ -z "$1" ] ; then
 fi
 CROSS_TARGET=$1
 BUILDARGS=''
-M3CONFIG=${ROOT}/m3-sys/cm3/src/config/${CROSS_TARGET}
+M3CONFIG=${ROOT}/m3-sys/cminstall/src/config-no-install/${CROSS_TARGET}
 export M3CONFIG
 . "$ROOT/scripts/pkginfo.sh"
 . "$ROOT/scripts/pkgcmds.sh"
@@ -38,25 +38,7 @@ P="${P} m3back"
 P="${P} m3front"
 P="${P} m3quake"
 P="${P} cm3"
-P="${P} m3scanner"
-P="${P} m3tools"
-P="${P} m3cgcat"
-P="${P} m3cggen"
-[ "${M3GDB}" = yes ] && P="${P} m3gdb"
 P="${P} m3bundle"
-P="${P} mklib"
-P="${P} fix_nl"
-P="${P} libdump"
-P="${P} bitvector"
-P="${P} digraph"
-P="${P} parseparams"
-P="${P} realgeometry"
-P="${P} set"
-P="${P} slisp"
-P="${P} sortedtableextras"
-P="${P} table-list"
-P="${P} tempfiles"
-[ "${HAVE_TCL}" = "yes" ] && P="${P} tcl"
 
 USAGE="
   `basename $0` cross_target

From jay.krell at cornell.edu  Sun Jan 19 12:35:54 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:35:54 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBA2EF.4020101@marino.st>
References: <52DBA2EF.4020101@marino.st>
Message-ID: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>

C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/013940cc/attachment.html>

From jay.krell at cornell.edu  Sun Jan 19 12:40:17 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:40:17 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>

Your change looks about right.
 
Did you first "upgrade" and existing install, with these changes, and then cross?
That is what you must do.
 
 - Jay
 
From: jay.krell at cornell.edu
To: adacore at marino.st; m3devel at elegosoft.com
Subject: RE: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Date: Sun, 19 Jan 2014 11:35:54 +0000




C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/1e7c8efe/attachment.html>

From adacore at marino.st  Sun Jan 19 12:53:53 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 12:53:53 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <52DBBCD1.7020904@marino.st>

On 1/19/2014 12:35, Jay K wrote:
> C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
> m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized
> target machin
> e: TARGET = ", s.target);
>  
>  
>  Please don't bother with the last release.
>  Please use the current CVS version.

But the last release is what is in FreeBSD ports, the basis of both
FreeBSD and DragonFly.  And ports doesn't build from repos normally,
although there is a facility to pull directly from github, but I don't
like that method myself.

If there was an official downloadable snapshot or release candidate, I
could use that.  Since 2010 was the last release, is another one coming
soon?


>  And please use the C backend.
>  Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?

Yes, that was one of the many patches in the attachment I send in the
previous mail.


>  Please use this as a guide for files to consider changing:
>  C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
> m3-comm\udp\test\src\m3makefile
> m3-libs\libm3\src\os\POSIX\m3makefile
> m3-libs\m3core\src\float\m3makefile
> m3-libs\m3core\src\thread\m3makefile
> m3-libs\m3core\src\thread\PTHREAD\m3makefile
> m3-libs\m3core\src\unix\m3makefile
> m3-sys\m3cc\src\m3makefile
> m3-sys\m3gdb\src\m3makefile
> m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
> m3-tools\cvsup\suplib\src\m3makefile
> m3-tools\cvsup\suptcp\src\POSIX\m3makefile

The only ones I haven't patched already are
m3-libs\m3core\src\thread\PTHREAD\m3makefile and the cvsup ones.  Those
wouldn't cause the error.  I also assume this is the top of the repo.


> C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
>   m3-libs\libm3\src\uid\POSIX\getMID.c 
>   m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c
>   m3-libs\m3core\src\Csupport\Common\s_lroundl.c
>  m3-libs\m3core\src\float\C99\FloatModeC.c 
>    m3-libs\m3core\src\runtime\common\RTProcessC.c 
>   m3-libs\m3core\src\runtime\POSIX\RTSignalC.c  
>   m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c
>   m3-libs\m3core\src\unix\Common\Uin.c
>   m3-libs\patternmatching\src\libglob\fnmatch.c
> 
>  
> esp. m3core and libm3.

I haven't seen freebsd in these, so must be a difference between release
and repo.



>  If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it
> for you.
>  But I realize there is much value in me NOT doing it.
>  I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok.

Because AMD64_DRAGONFLY is too long?
I'd prefer to keep it "AMD64_DRAGONFLY" because it matches the other
targets and because of all the work I've already done.


I'm really, really close to getting cm3 bootstrapped on DragonFly with
5.8.6.  I believe if this "M3_BOOTSTRAP = true" version would compile, I
could build cm3 and libraries on DragonFly with the cm3cg that's already
built on it.  Once I have a fully native compiler, then I can update to
the latest repo, especially if some kind of snapshot is released.

Thanks,
John


From jay.krell at cornell.edu  Sun Jan 19 13:00:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:00:01 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
Message-ID: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>

Windows uses SuspendThread.
I'll double check.


Cooperative suspend would NOT involve sending a "signal" to threads, in the Posix sense.
Nor would it use SuspendThread.


The model for cooperative suspend has been described by Tony and goes something like this:

There is a list of threads maintained by the m3 runtime.
  Not by enumerating OS threads.
The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.


When a thread wants to collect garbage, it either walks the thread list and sets all the booleans/enums to "suspend requested", or, more likely, it sets one global boolean.

Now, there are three likely methods to set the global boolean.
 1 - it is a global writable variable 
 2 - it is a global writable function pointer 
 3 - it is a page with protection 


Code frequently polls this global boolean, in one of the following fashions
 1 - read and test the value of the variable 
 2 - call the function pointer 
 3 - read the page (just a byte or a word) ignoring the contents 
 4 - call a function to do any of the above, esp #1 


#3 is probably fastest but my least favorite. It has lingering portability problems and is slowest when it comes time to actually suspend -- catch the exception and for purposes of debugging, you really don't want a design that raises exceptions like that.


When does the polling occur?
The polling is generated by the compiler.
It is done at function entry and/or exit and at branches or at least backward branches or at least branches that are continuing loops or such or at the starts of loops or somesuch.


I believe Java uses cooperative suspend and I think it uses the page protection mechanism.
It isn't portable e.g. to systems without an MMU.
Though of course you could abstract it out.


I'm glossing over details, but you know all the code we have for signals and semaphores? It goes away.
Our portability then would drive one more significant notch toward that of "hello world".

In my mine, we have about two pieces to do to gain "hello world" level of portability
  - cooperative suspend, not easy  
  - jmpbuf size removal, should be easy  

and some small matters
  - endianness, particularly in the libary 
  - word size 



 - Jay


From: dragisha at m3w.org
Date: Sat, 18 Jan 2014 09:24:11 +0100
To: m3devel at elegosoft.com
Subject: [M3devel] Cooperative suspend?

Jay,
What exactly do you mean? Isn't it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?
dd

--Dragi?a Duri?dragisha at m3w.org



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/88389edf/attachment.html>

From adacore at marino.st  Sun Jan 19 13:09:37 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:09:37 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
Message-ID: <52DBC081.1070401@marino.st>

On 1/19/2014 12:40, Jay K wrote:
> Your change looks about right.
>  
> Did you first "upgrade" and existing install, with these changes, and
> then cross?
> That is what you must do.


It's probably easier that I show you.
I've attached "Makefile.local" (I added the .txt extension just now to
help out some mail clients) which is my recipe.

So I run three targets in a freshly patched workarea:
1) make cross-compiler
2) make assembler-code (called from cross-archive)
3) make cross-archive

In the cross-compiler target, the FreeBSD bootstrap compiler builds
m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.

The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
Everything is cleaned, then the cross compiler builds everything again
(including libm3core and libm3) along with m3objfile, sysutils, and
patternmatching.

Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
cm3cg is cm3cg-AMD64_DRAGONFLY.

I would not be surprised is there is a logic flaw or two in "make
cross-compiler".  The "how to port CM3" tutorial was slightly out of
date and some of the instructions were a bit ambiguous so this was a
best guess.

Regards,
John
-------------- next part --------------
# This makefile fragment is automatically included by bsd.port.mk
# It is separate because it contains maintenance targets not used in
# normal package building, but the recipes are too value to lose.  So
# we keep them with the port, but out of the way of regular users.
#
# $FreeBSD$


##
#
# The "create-bootstrap" target will assemble a new bootstrap based on the
# contents of the stage directory, which means it has to be run after the
# "stage" target.
#
##

NEWBOOTDIR=	${WRKDIR}/new-bootstrap/bootstrap
NEWBOOTNAME=	m3-bootstrap.${MARCH}.${OPSYS:U}.${OSREL:S/.//}.tar.bz2
BSCONTENTS=	bin/cm3 bin/cm3cg bin/m3bundle bin/mklib etc/modula3 \
		lib/libm3core.* lib/libm3.* lib/libsysutils.* \
		lib/libpatternmatching.* pkg/m3core pkg/libm3 pkg/sysutils \
		pkg/patternmatching pkg/m3middle pkg/m3objfile pkg/m3linker \
		pkg/m3back pkg/m3front pkg/m3quake pkg/cm3 pkg/mklib

create-bootstrap:
	@${RM} -rf ${NEWBOOTDIR}
	@${MKDIR} ${NEWBOOTDIR}/bin ${NEWBOOTDIR}/lib \
	          ${NEWBOOTDIR}/pkg ${NEWBOOTDIR}/etc
.for X in ${BSCONTENTS}
	@${CP} -a ${STAGEDIR}${PREFIX}/${X} ${NEWBOOTDIR}/${X:H}/
.endfor
	${ECHO} "INSTALL_ROOT = path() & \"/..\"" \
		>  ${NEWBOOTDIR}/bin/cm3.cfg
	${ECHO} "include(path() & \"/../etc/modula3/${M3TARGET}\")" \
		>> ${NEWBOOTDIR}/bin/cm3.cfg
	@${FIND} ${NEWBOOTDIR} -type f -perm +111 | ${XARGS} ${STRIP_CMD}
	(cd ${NEWBOOTDIR}/.. ; tar -cyf ${NEWBOOTNAME} bootstrap)


##
#
# The "cross-compiler" target will create a cross-compiler in the
# stage directory based on the value of ${CROSSTARG}.
# Execute it after the "patch" target
#
# The "cross-archive" will archive assembler code meant for linking
# on the native plaform.  It is run after "cross-compiler"
#
# The "native-link" target will bootstrap cm3 on a new platform (which
# obviously supports ports) using the archived files generated above.
# It could be done remotely via SSH, but this one target assumes files
# are locally available at ${WRKSRC}/${CROSSTARG}
#
##

CROSSTARG?=	AMD64_DRAGONFLY
CORELIBS=	m3-libs/m3core m3-libs/libm3
CORESYS=	m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker \
		m3-sys/m3back m3-sys/m3front m3-sys/m3quake m3-sys/cm3 \
		m3-tools/m3bundle
COREHELPERS=	m3-libs/sysutils m3-libs/patternmatching
COREFILES=	${CORELIBS} ${CORESYS}

cross-compiler:
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/build-cross-backend.sh ${CROSSTARG})
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/do-pkg.sh onlybuild \
		m3middle m3linker m3front m3quake cm3)
	@${FIND} ${WRKSRC} -name \.M3SHIP -print | ${XARGS} ${SED} -i -e \
		's|/bootstrap/|/stage${PREFIX}/|g'
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh ship \
		m3middle m3linker m3front m3quake cm3)	
	(cd ${WRKSRC}/m3-sys/cminstall/src/config-no-install && \
		${COPYTREE_SHARE} . ${STAGEDIR}${PREFIX}/etc/modula3)
	${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/cm3/${M3TARGET}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${M3TARGET}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${INSTALL_PROGRAM} ${WRKDIR}/bootstrap/bin/cm3cg \
		${STAGEDIR}${PREFIX}/bin/
	${INSTALL_PROGRAM} \
		${WRKSRC}/m3-sys/m3cc/${M3TARGET}-${CROSSTARG}/cm3cg \
		${STAGEDIR}${PREFIX}/bin/cm3cg-${CROSSTARG}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal \
		m3middle m3linker m3front m3quake cm3)
.for component in ${CORELIBS}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T})
.endfor
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${CORELIBS:T})
.for component in ${CORELIBS} ${COREHELPERS} ${CORESYS}
	@(cd ${WRKSRC}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${component:T} \
	)
.endfor
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "m3back = INSTALL_ROOT & \"/bin/cm3cg-${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "BUILD_DIR = \"${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "HOST = \"${M3TARGET}\"" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${CROSSTARG}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "M3_BOOTSTRAP = TRUE" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	@${ECHO} "====================================="
	@${ECHO} "=====  M3 cross-compiler ready  ====="
	@${ECHO} "====================================="
	@${ECHO}

assembler-code:
.for component in ${COREFILES}
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} SHIP=${TRUE} TARGET=${CROSSTARG} \
	    ${BUILTCM3} ${SH} scripts/do-pkg.sh onlybuild ${component:T} \
	)
.endfor
.for component in m3-libs/libm3 m3-sys/m3middle
	echo '#include "m3core.h"' > \
		${WRKSRC}/${component}/${CROSSTARG}/m3unix.h
.endfor
	${CP} ${WRKSRC}/m3-libs/m3core/src/m3core.h \
		${WRKSRC}/m3-libs/libm3/${CROSSTARG}/
	(cd ${WRKSRC}/m3-sys/m3middle; ${CP} src/POSIX/m3core.h ${CROSSTARG}/)

cross-archive: assembler-code
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} M3GDB=no ${BUILTCM3} \
		${SH} scripts/pack-crossbuild.sh ${CROSSTARG})

native-cm3cg:
# This target requires python to be installed
	@(cd ${WRKSRC}/m3-sys/m3cc; src/boot.py)
	@${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/m3cc/obj.1/gcc/m3cgc1 \
		${STAGEDIR}${PREFIX}/bin/cm3cg

native-link: native-cm3cg
.for component in ${COREFILES}
	@(cd ${WRKSRC}/${CROSSTARG}; ${TAR} -xzf ${component:T}.tgz)
.endfor
	@(cd ${WRKSRC}/scripts; \
		${SETENV} PATH=${PATH}:${STAGEDIR}${PREFIX}/bin \
		./boot-cm3-build-on-target.sh ${CROSSTARG})
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-sys/cm3/${CROSSTARG}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-tools/m3bundle/${CROSSTARG}/m3bundle \
		${STAGEDIR}${PREFIX}/bin

From jay.krell at cornell.edu  Sun Jan 19 13:17:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:17:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC081.1070401@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
Message-ID: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>

Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/0b2d21ba/attachment.html>

From adacore at marino.st  Sun Jan 19 13:38:32 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:38:32 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <52DBC748.5090308@marino.st>

On 1/19/2014 13:17, Jay K wrote:
> Hey..I like that that does capture some of My Way. 
> Here is my independent off the cuff write up though:
>  I know the ports system is nice.
>  I know we really need to be in it for many operating systems.
>  I suspect your problem is that you didn't "upgrade" to apply you
> changes at the right time.

I must be misunderstanding what you mean by "upgrade".  It's the
cross-compiler that's complaining, and it was built by the latest patches.


>  Do any of these work for you:
>  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html

I'm not clear to which set you are referring.
There's no need for a binary set, modula3 now builds on FreeBSD 9 and
FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
superior to the release binaries because they link gmp and mpfr
statically while a bug in the cm3 sources caused those libraries to link
dynamically (which is why release binaries don't run on current FreeBSD)

Then there are the source distributions, but they are from 2012 so they
aren't exactly fresh either.


>  Anyway, please try this: 
>   Ignore DragonFly at first. 
>   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> anything. 
>   Such as from ports. 
>   Make a backup, e.g. of /usr/local/cm3. 
>   My instructions are unfortunately destructive of the existing install. 
>   Checkout the full current CVS repository. 
>   In that repository, run scripts/python/upgrade.py. 
>   If that doesn't work, stop, and we'll help. 

Is the goal just to get FreeBSD to have a binary with the latest
sources?  If so, I'd just modify the port to do that now that we have
working binary bootstraps.  The main reason I'm reluctant is that it
took over a week to make the cm3 port, I've never had so much difficulty
before.  (I've also spent another week on the DragonFly port).  So I'm
"resisting" because of how much work it already took.  That's why I was
trying to port DragonFly, THEN update to latest rather than update to
latest and then port DragonFly.  (in other words, going forward seems a
short path than backtracking)



>   If that does work, apply your diffs. And upgrade.py again. You can say
> "upgrade.py skipgcc" here. 
>   Then boot1.py c amd64_dragonflybsd   
>     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> on the command line. 
>     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> gcc/amd64 works for the same
>     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.

I assume you mean after the system understands this target.  There will
be a lot of files with __FreeBSD__ that need to be changed to
__FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
regenerate a bunch of patches from my current work.


>   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
>   Copy that to your Dragonfly system, extract it, cd into it, make. 
>   If that works, you have a native cm3. 
>   Run it. It should say "can't find cm3.cfg".  
>   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
>   cp cm3 /usr/local/cm3/bin  
>   export PATH=/usr/local/cm3/bin:$PATH
>   full CVS checkout  
>   cd scripts/python ; ./boot2.sh 
>    
>   You will need to have made small changes to pylib.py for this to work.
>   (And your config file should say to use the C backend, like Darwin and
> ARM_LINUX do.)


Well, I'm thrilled if I don't have to patch gcc again, that takes a long
time.  Shouldn't FreeBSD use the C backend too?

John


From jay.krell at cornell.edu  Sun Jan 19 13:32:44 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:32:44 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>, ,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>,
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>, ,
	<52DBC081.1070401@marino.st>,
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <COL130-W171FDF068BFCF6E20D357EE6BA0@phx.gbl>

Clarification:
upgrade.sh (note the .sh) rebuilds everything
I wrote upgrade.py based on it, but missed the "everything" part.
upgrade.py only upgrades "the compiler" (and m3core, libm3..)

A better practise although inefficient is:

 upgrade.py && ./do-cm3-all.py realclean skipgcc && ./do-cm3-all.py buildship  

I didn't suggest that to you here in order to get further in the pipeline faster.

However upgrade.py w/o other rebuild could leave m3core.so too new for many programs.
But for purposes of building/cross-building just the compiler, my instructions are good.


John, I see you added cm3 to the FreeBSD ports and you clearly know what you are doing.
Nevertheless, I am a lazy snob, don't quite see what you are doing wrong, and am suggesting "just different", not clearly better.
But, to my credit, this "just different" is what I use all the time.
I have used it to cross to several systems, w/ and w/o cm3cg (w/ and w/o C backend).
It isn't fast, and it doesn't always have the best incrementality, but it does usually work.
There is redundancy. pylib.py essentially duplicates parts of the config files.


You misunderstood what I meant by "C backend" and from the ports history you will somewhat appreciate it.
The C backend, instead of having a lot to do with gcc, is a backend that generates C, and then invokes a C compiler.
The problem you had with -gstabs for example, goes away. We'll use whatever debugging format regular C/C++ use.
There is a little hope of taking the one C output and compiling it for different targets -- at least FreeBSD 8,9,10.
Currently the C is very word size dependent, endian dependent, and somewhat otherwise target-dependent.
I'd like to fix these aspects but it isn't easy by far, will take possibly controversial changes in the frontend and interface to backend.


 - Jay




From: jay.krell at cornell.edu
To: adacore at marino.st
Date: Sun, 19 Jan 2014 12:17:24 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)




Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/153439e7/attachment.html>

From jay.krell at cornell.edu  Sun Jan 19 13:50:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:50:27 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC748.5090308@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
Message-ID: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>

I understand your reluctance.


> Is the goal just to get FreeBSD to have a binary with the latest
> sources?


Yes, and to make sure the build process works for you.


> Shouldn't FreeBSD use the C backend too?


Every target should. I haven't convinced everyone yet.
There is less resistance for new targets, like AMD64_NT and ARM_LINUX, which are working, using the C backend.
There are downsides to the C backend:
  - it is slower to compiler  
  - debugging is degraded on systems that have m3gdb  
 (debugging is much better on systems w/o m3gdb) 


And there are advantages not yet implemented, in particular, efficient exception handling by generating C++.


> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.


Your patches should apply fairly cleanly to any new checkout.
I am willing to through them and apply them to CVS if you want -- depends on if you are ok with my stealing your credit, vs. you want to commit them yourselves.
I already skimmed them and they all look easy.
DragonflyBSD is basically FreeBSD by another name.
Yes, a ton of work has been done on the kernel, file systems, maybe ports.
But the ABI is almost the same -- heck, for the most part, Linux==NetBSD==FreeBSD==OpenBSD.
Sure, they might be implemented differently, but they are highly source compatible and every highly object code compatible.


> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.

Agreed, we are both missing something simple.
Usually this is the error you get when you don't upgrade Target.i3/Target.m3.
If you want to press on with the current approach and ignore my advise, it should be possible to step through all this.
To start, break on Target__Init and see if it returns true or false.


> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)


Ugh. I saw something about that in your diffs. Definitely we intend to link statically.
I actually went to the trouble of removing use of gmp/mpfr/mpc in current source.
They are used for actually very little and aren't worth it.


 - Jay


> Date: Sun, 19 Jan 2014 13:38:32 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 13:17, Jay K wrote:
> > Hey..I like that that does capture some of My Way. 
> > Here is my independent off the cuff write up though:
> >  I know the ports system is nice.
> >  I know we really need to be in it for many operating systems.
> >  I suspect your problem is that you didn't "upgrade" to apply you
> > changes at the right time.
> 
> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.
> 
> 
> >  Do any of these work for you:
> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
> 
> I'm not clear to which set you are referring.
> There's no need for a binary set, modula3 now builds on FreeBSD 9 and
> FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
> superior to the release binaries because they link gmp and mpfr
> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)
> 
> Then there are the source distributions, but they are from 2012 so they
> aren't exactly fresh either.
> 
> 
> >  Anyway, please try this: 
> >   Ignore DragonFly at first. 
> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> > anything. 
> >   Such as from ports. 
> >   Make a backup, e.g. of /usr/local/cm3. 
> >   My instructions are unfortunately destructive of the existing install. 
> >   Checkout the full current CVS repository. 
> >   In that repository, run scripts/python/upgrade.py. 
> >   If that doesn't work, stop, and we'll help. 
> 
> Is the goal just to get FreeBSD to have a binary with the latest
> sources?  If so, I'd just modify the port to do that now that we have
> working binary bootstraps.  The main reason I'm reluctant is that it
> took over a week to make the cm3 port, I've never had so much difficulty
> before.  (I've also spent another week on the DragonFly port).  So I'm
> "resisting" because of how much work it already took.  That's why I was
> trying to port DragonFly, THEN update to latest rather than update to
> latest and then port DragonFly.  (in other words, going forward seems a
> short path than backtracking)
> 
> 
> 
> >   If that does work, apply your diffs. And upgrade.py again. You can say
> > "upgrade.py skipgcc" here. 
> >   Then boot1.py c amd64_dragonflybsd   
> >     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> > on the command line. 
> >     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.
> 
> 
> >   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
> >   Copy that to your Dragonfly system, extract it, cd into it, make. 
> >   If that works, you have a native cm3. 
> >   Run it. It should say "can't find cm3.cfg".  
> >   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
> >   cp cm3 /usr/local/cm3/bin  
> >   export PATH=/usr/local/cm3/bin:$PATH
> >   full CVS checkout  
> >   cd scripts/python ; ./boot2.sh 
> >    
> >   You will need to have made small changes to pylib.py for this to work.
> >   (And your config file should say to use the C backend, like Darwin and
> > ARM_LINUX do.)
> 
> 
> Well, I'm thrilled if I don't have to patch gcc again, that takes a long
> time.  Shouldn't FreeBSD use the C backend too?
> 
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/ce74cb6a/attachment.html>

From adacore at marino.st  Sun Jan 19 14:06:02 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 14:06:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <52DBCDBA.8030809@marino.st>

On 1/19/2014 13:50, Jay K wrote:
> And there are advantages not yet implemented, in particular, efficient
> exception handling by generating C++.

Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
dl_iterate_phdr which gcc uses for efficient zero-cost exception
handling.  That might come in handy here (I implemented it in
DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)


> Your patches should apply fairly cleanly to any new checkout.
> I am willing to through them and apply them to CVS if you want --
> depends on if you are ok with my stealing your credit, vs. you want to
> commit them yourselves.

I would love it if you did this - I'm happy with a mention in the commit
message.  I just attached a tarball of all the patches I'm using right
now.  Some of these changes are specifically for the cross compiler (I'm
think of the /scripts changes mostly) but most are valid assuming they
still apply to the head of the repo.  Please take as much as you want
with my gratitude.

> I already skimmed them and they all look easy.
> DragonflyBSD is basically FreeBSD by another name.

That's not an accident.  As a DragonFly developer that mostly works on
userland, I've been converging FreeBSD and DragonFly again.  They
drifted apart after 10 years, but I thought it better to keep the
userlands as similar as possible for 3rd party applications.

> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> But the ABI is almost the same -- heck, for the most part,
> Linux==NetBSD==FreeBSD==OpenBSD.
> Sure, they might be implemented differently, but they are highly source
> compatible and every highly object code compatible.
>> I must be misunderstanding what you mean by "upgrade". It's the
>> cross-compiler that's complaining, and it was built by the latest patches.
> Agreed, we are both missing something simple.
> Usually this is the error you get when you don't upgrade
> Target.i3/Target.m3.
> If you want to press on with the current approach and ignore my advise,

I'm not ignoring it; it's more like I'm not quite ready to give up since
I'm so close.  I think I could actually get these libs and cm3 to build
on DragonFly if I used AMD64_FREEBSD as the target with the
cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
I'd then get stuck on the DragonFly side trying to compile cm3 with
itself, theoretically I should get the same error there.


> Ugh. I saw something about that in your diffs. Definitely we intend to
> link statically.
> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> current source.
> They are used for actually very little and aren't worth it.

Even better!
Regards,
John

-------------- next part --------------
A non-text attachment was scrubbed...
Name: files.tar
Type: application/x-tar
Size: 13062 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/4ce4f917/attachment.tar>

From dragisha at m3w.org  Sun Jan 19 15:28:18 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 19 Jan 2014 15:28:18 +0100
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
Message-ID: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>

Why would we do this, and not signalling, as in POSIX implementation?

Why try to make something synchronous when we have working asynchronous implementation?

Why would it be better than sync one?

On 19 Jan 2014, at 13:00, Jay K <jay.krell at cornell.edu> wrote:

> There is a list of threads maintained by the m3 runtime.
>   Not by enumerating OS threads.
> The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment.sig>

From mika at async.caltech.edu  Sun Jan 19 21:05:48 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:05:48 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
Message-ID: <20140119200548.C18011A208F@async.async.caltech.edu>

"less worried" = "I think it works"?

I think the only threading that works in CM3 is user threading...

I tried to run CM3-compiled binaries through "valgrind" but got an
error that CM3 is using the same signal that valgrind does internally
(on AMD64_LINUX at least).  I think this is easy to change in m3core
(yes?) but haven't gotten around to it...

valgrind has a threading checker called "helgrind"...

     Mika

Jay writes:
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/plain;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>w up further.
>
> - Jay
>
>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>
>
>> Jay:
>> =20
>> I=E2=80=99m very concerned about the threading not working properly on bot=
>h 32-bit and 64-bit Windows.
>> =20
>> The Thread Test program crashes for me on both platforms.
>> =20
>> I haven=E2=80=99t tried your new test program yet.
>> =20
>> --Randy
>> =20
>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>> Sent: Saturday, January 18, 2014 1:23 AM
>> To: m3devel
>> Subject: EXT:[M3devel] Win32 SuspendThread?
>> =20
>>  This program also doesn't behave as expected, native, nothing to do with w=
>ow64.=20
>>  Anyone else please confirm:=20
>>    1) my expectations -- it should never print anything
>>    2) their importance  -- garbage collector depends on it =20
>>    3) their not being met -- stuff gets printed=20
>>  This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>  I am following up further.=20
>>  Maybe we should get cooperative suspend really going?=20
>> Thank you.
>>  - Jay=20
>> =20
>> #include <stdio.h>
>> #include <windows.h>
>> volatile long value;
>> unsigned long __stdcall Thread(PVOID parameter)
>> {
>>   while (1)
>>    InterlockedIncrement(&value);
>>   return 0;
>> }
>> int __cdecl main()
>> {
>>   HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>   UINT i =3D 0;
>>   while (1)
>>   {
>>     i +=3D 1;
>>     if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>     {
>>       printf("suspend failed %X\n", GetLastError());
>>       Sleep(1);
>>       continue;
>>     }
>>     volatile long a =3D value;=20
>>     volatile long b =3D value;
>>     if (a !=3D b)
>>     {
>>      printf("%d %d %d %d\n", i, a, b, b - a);
>>     }
>>     ResumeThread(thread);
>>   }
>> }
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/html;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
><blockquote type=3D"cite"><div>
>
><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>
><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
><style><!--
>/* Font Definitions */
>@font-face
>	{font-family:Calibri;
>	panose-1:2 15 5 2 2 2 4 3 2 4;}
>@font-face
>	{font-family:Tahoma;
>	panose-1:2 11 6 4 3 5 4 4 2 4;}
>/* Style Definitions */
>p.MsoNormal, li.MsoNormal, div.MsoNormal
>	{margin:0in;
>	margin-bottom:.0001pt;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>a:link, span.MsoHyperlink
>	{mso-style-priority:99;
>	color:blue;
>	text-decoration:underline;}
>a:visited, span.MsoHyperlinkFollowed
>	{mso-style-priority:99;
>	color:purple;
>	text-decoration:underline;}
>p
>	{mso-style-priority:99;
>	mso-margin-top-alt:auto;
>	margin-right:0in;
>	mso-margin-bottom-alt:auto;
>	margin-left:0in;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>	{mso-style-priority:99;
>	mso-style-link:"Balloon Text Char";
>	margin:0in;
>	margin-bottom:.0001pt;
>	font-size:8.0pt;
>	font-family:"Tahoma","sans-serif";}
>span.EmailStyle18
>	{mso-style-type:personal-reply;
>	font-family:"Calibri","sans-serif";
>	color:#1F497D;}
>span.BalloonTextChar
>	{mso-style-name:"Balloon Text Char";
>	mso-style-priority:99;
>	mso-style-link:"Balloon Text";
>	font-family:"Tahoma","sans-serif";}
>.MsoChpDefault
>	{mso-style-type:export-only;
>	font-size:10.0pt;}
>@page WordSection1
>	{size:8.5in 11.0in;
>	margin:1.0in 1.0in 1.0in 1.0in;}
>div.WordSection1
>	{page:WordSection1;}
>--></style><!--[if gte mso 9]><xml>
><o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
></xml><![endif]--><!--[if gte mso 9]><xml>
><o:shapelayout v:ext=3D"edit">
><o:idmap v:ext=3D"edit" data=3D"1" />
></o:shapelayout></xml><![endif]-->
>
>
><div class=3D"WordSection1">
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>shes for me on both platforms.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>r new test program yet.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><div>
><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>in 0in">
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
><b>On Behalf Of </b>Jay K<br>
><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
><b>To:</b> m3devel<br>
><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
></div>
></div>
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
><div>
><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>in-bottom:12.0pt;margin-left:.5in">
><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>;This program also doesn't behave as expected, native, nothing to do with wo=
>w64.
><br>
>&nbsp;Anyone else please confirm: <br>
>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>sp; <br>
>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>&nbsp;I am following up further. <br>
>&nbsp;Maybe we should get cooperative suspend really going? <br>
>Thank you.<br>
>&nbsp;- Jay <br>
>&nbsp;<br>
>#include &lt;stdio.h&gt;<br>
>#include &lt;windows.h&gt;<br>
>volatile long value;<br>
>unsigned long __stdcall Thread(PVOID parameter)<br>
>{<br>
>&nbsp; while (1)<br>
>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>&nbsp; return 0;<br>
>}<br>
>int __cdecl main()<br>
>{<br>
>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>&nbsp; UINT i =3D 0;<br>
>&nbsp; while (1)<br>
>&nbsp; {<br>
>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>;<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>&nbsp; }<br>
>}<o:p></o:p></span></p>
></div>
></div>
>
>
></div></blockquote></body></html>=
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From mika at async.caltech.edu  Sun Jan 19 21:13:08 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:13:08 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <20140119201308.0A4B41A208F@async.async.caltech.edu>


There were some bugs we never got around to addressing with the C backend,
weren't there?  A crash that didn't happen with native backend.

I've been using the "native" at more or less the head on very recent
Debian and the Raspberry Pi for a while now with no problems to speak of.
(As long as you only use user threads... but that's another story, I think.)

     Mika

Jay K writes:
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>I understand your reluctance.
>
>
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?
>
>
>Yes=2C and to make sure the build process works for you.
>
>
>> Shouldn't FreeBSD use the C backend too?
>
>
>Every target should. I haven't convinced everyone yet.
>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>which are working=2C using the C backend.
>There are downsides to the C backend:
>  - it is slower to compiler =20
>  - debugging is degraded on systems that have m3gdb =20
> (debugging is much better on systems w/o m3gdb)=20
>
>
>And there are advantages not yet implemented=2C in particular=2C efficient =
>exception handling by generating C++.
>
>
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>
>
>Your patches should apply fairly cleanly to any new checkout.
>I am willing to through them and apply them to CVS if you want -- depends o=
>n if you are ok with my stealing your credit=2C vs. you want to commit them=
> yourselves.
>I already skimmed them and they all look easy.
>DragonflyBSD is basically FreeBSD by another name.
>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>orts.
>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>Sure=2C they might be implemented differently=2C but they are highly source=
> compatible and every highly object code compatible.
>
>
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>
>Agreed=2C we are both missing something simple.
>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>3.
>If you want to press on with the current approach and ignore my advise=2C i=
>t should be possible to step through all this.
>To start=2C break on Target__Init and see if it returns true or false.
>
>
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>
>
>Ugh. I saw something about that in your diffs. Definitely we intend to link=
> statically.
>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>ource.
>They are used for actually very little and aren't worth it.
>
>
> - Jay
>
>
>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>> From: adacore at marino.st
>> To: jay.krell at cornell.edu
>> CC: m3devel at elegosoft.com
>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>=20
>> On 1/19/2014 13:17=2C Jay K wrote:
>> > Hey..I like that that does capture some of My Way.=20
>> > Here is my independent off the cuff write up though:
>> >  I know the ports system is nice.
>> >  I know we really need to be in it for many operating systems.
>> >  I suspect your problem is that you didn't "upgrade" to apply you
>> > changes at the right time.
>>=20
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>>=20
>>=20
>> >  Do any of these work for you:
>> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>=20
>> I'm not clear to which set you are referring.
>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>> superior to the release binaries because they link gmp and mpfr
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>>=20
>> Then there are the source distributions=2C but they are from 2012 so they
>> aren't exactly fresh either.
>>=20
>>=20
>> >  Anyway=2C please try this:=20
>> >   Ignore DragonFly at first.=20
>> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>> > anything.=20
>> >   Such as from ports.=20
>> >   Make a backup=2C e.g. of /usr/local/cm3.=20
>> >   My instructions are unfortunately destructive of the existing install=
>.=20
>> >   Checkout the full current CVS repository.=20
>> >   In that repository=2C run scripts/python/upgrade.py.=20
>> >   If that doesn't work=2C stop=2C and we'll help.=20
>>=20
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?  If so=2C I'd just modify the port to do that now that we have
>> working binary bootstraps.  The main reason I'm reluctant is that it
>> took over a week to make the cm3 port=2C I've never had so much difficult=
>y
>> before.  (I've also spent another week on the DragonFly port).  So I'm
>> "resisting" because of how much work it already took.  That's why I was
>> trying to port DragonFly=2C THEN update to latest rather than update to
>> latest and then port DragonFly.  (in other words=2C going forward seems a
>> short path than backtracking)
>>=20
>>=20
>>=20
>> >   If that does work=2C apply your diffs. And upgrade.py again. You can =
>say
>> > "upgrade.py skipgcc" here.=20
>> >   Then boot1.py c amd64_dragonflybsd  =20
>> >     Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>re
>> > on the command line.=20
>> >     Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>=2C
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>>=20
>>=20
>> >   If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>gz=20
>> >   Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>e.=20
>> >   If that works=2C you have a native cm3.=20
>> >   Run it. It should say "can't find cm3.cfg". =20
>> >   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>> >   cp cm3 /usr/local/cm3/bin =20
>> >   export PATH=3D/usr/local/cm3/bin:$PATH
>> >   full CVS checkout =20
>> >   cd scripts/python =3B ./boot2.sh=20
>> >   =20
>> >   You will need to have made small changes to pylib.py for this to work=
>.
>> >   (And your config file should say to use the C backend=2C like Darwin =
>and
>> > ARM_LINUX do.)
>>=20
>>=20
>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>ong
>> time.  Shouldn't FreeBSD use the C backend too?
>>=20
>> John
> 		 	   		  =
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 12pt=3B
>font-family:Calibri
>}
>--></style></head>
><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>h are working=2C using the C backend.<br>There are downsides to the C backe=
>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>et implemented=2C in particular=2C efficient exception handling by generati=
>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>=3B     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>=3B <br>&gt=3B I assume you mean after the system understands this target. =
> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'=
>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
> am willing to through them and apply them to CVS if you want -- depends on=
> if you are ok with my stealing your credit=2C vs. you want to commit them =
>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>e highly source compatible and every highly object code compatible.<br><br>=
><br>&gt=3B I must be misunderstanding what you mean by "upgrade".  It's the=
><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>r>If you want to press on with the current approach and ignore my advise=2C=
> it should be possible to step through all this.<br>To start=2C break on Ta=
>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
> to link statically.<br>I actually went to the trouble of removing use of g=
>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>dependent off the cuff write up though:<br>&gt=3B &gt=3B  I know the ports =
>system is nice.<br>&gt=3B &gt=3B  I know we really need to be in it for man=
>y operating systems.<br>&gt=3B &gt=3B  I suspect your problem is that you d=
>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade".  =
>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Do any of thes=
>e work for you:<br>&gt=3B &gt=3B  https://modula3.elegosoft.com/cm3/snaps/s=
>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Anyway=2C plea=
>se try this: <br>&gt=3B &gt=3B   Ignore DragonFly at first. <br>&gt=3B &gt=
>=3B   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B   Such as from ports. <br>&gt=3B =
>&gt=3B   Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B   My in=
>structions are unfortunately destructive of the existing install. <br>&gt=
>=3B &gt=3B   Checkout the full current CVS repository. <br>&gt=3B &gt=3B   =
>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B   If=
> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>?  If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>working binary bootstraps.  The main reason I'm reluctant is that it<br>&gt=
>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>lty<br>&gt=3B before.  (I've also spent another week on the DragonFly port)=
>.  So I'm<br>&gt=3B "resisting" because of how much work it already took.  =
>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>t rather than update to<br>&gt=3B latest and then port DragonFly.  (in othe=
>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B   If that does work=2C appl=
>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>y skipgcc" here. <br>&gt=3B &gt=3B   Then boot1.py c amd64_dragonflybsd   <=
>br>&gt=3B &gt=3B     Make sure you say "c" and the target "amd64_dragonflyb=
>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B    =
> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B     for Linux=2C=
> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>me you mean after the system understands this target.  There will<br>&gt=3B=
> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to<br>&gt=3B =
>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>r>&gt=3B &gt=3B   If this works=2C it will give you=2C roughly amd64_dragon=
>flybsd*.tar.gz <br>&gt=3B &gt=3B   Copy that to your Dragonfly system=2C ex=
>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B   If that works=2C you ha=
>ve a native cm3. <br>&gt=3B &gt=3B   Run it. It should say "can't find cm3.=
>cfg".  <br>&gt=3B &gt=3B   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
><br>&gt=3B &gt=3B   cp cm3 /usr/local/cm3/bin  <br>&gt=3B &gt=3B   export P=
>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B   full CVS checkout  <br>&g=
>t=3B &gt=3B   cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B    <br>&gt=
>=3B &gt=3B   You will need to have made small changes to pylib.py for this =
>to work.<br>&gt=3B &gt=3B   (And your config file should say to use the C b=
>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>hat takes a long<br>&gt=3B time.  Shouldn't FreeBSD use the C backend too?<=
>br>&gt=3B <br>&gt=3B John<br></div> 		 	   		  </div></body>
></html>=
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_--


From mika at async.caltech.edu  Sun Jan 19 22:36:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 13:36:49 -0800
Subject: [M3devel] narrow still failing..
Message-ID: <20140119213649.403AE1A208F@async.async.caltech.edu>


A smaller test:

     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  PROCEDURE Z(VAR x : REFANY) =
    20    BEGIN
    21      x := BDD.Format(x)
    22    END Z;
    23
    24  BEGIN
    25    Q()
    26  END Main.

(191)rover:~/refany/src>../FreeBSD4/refany

***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 21
***

What's going on here???

Here are the relevant declarations:

(* simple BDD package *)

INTERFACE BDD;
IMPORT Word;

TYPE 
  T <: REFANY;

(* the boolean constants top and bottom *)
PROCEDURE True() : T;
PROCEDURE False() : T;

(* a new variable *)
PROCEDURE New(name : TEXT := NIL) : T;

(* unary ops *)
PROCEDURE Not(a : T) : T;

(* binary ops *)
PROCEDURE And(a, b : T) : T;
PROCEDURE Xor(a , b : T) : T;
PROCEDURE Equivalent(a, b : T) : T;
PROCEDURE Or( a, b : T) : T;
PROCEDURE Implies  (a , b : T) : T;

(* maketrue/makefalse *)

PROCEDURE MakeFalse(b, v : T) : T; (* make v false in b *)
PROCEDURE MakeTrue(b, v : T) : T; (* make v true in b *)


(* print with ids *)
PROCEDURE Format(a : T; symtab : REFANY (* BDDTextTbl.T *) := NIL) : TEXT;

(* the following procedures allow this interface to be used in generics *)
PROCEDURE Equal(a, b : T) : BOOLEAN;
PROCEDURE Hash(a : T) : Word.T;

PROCEDURE Size(a : T) : CARDINAL;
  (* number of nodes in structure *)

CONST
  Brand = "BDD 0.1";

PROCEDURE GetId(a : T) : INTEGER; (* for debugging *)

END BDD.
....

REVEAL
  T = BRANDED Brand OBJECT
    l , r : T;
    root : Root;
    tag : CARDINAL; (* for hashing *)
    name : TEXT;
  METHODS
    init() : T := Init;
  END;

PROCEDURE New(name : TEXT) : T =
  VAR res : Root := NEW(Root).init(); BEGIN
    res.name := name;
    res.root := res;
    res.l := true; res.r := false;
    res.id := nextId;
    res.tab := NEW(BDDTripleHash.Default).init(128);
    FOR i := FIRST(res.cache) TO LAST(res.cache) DO
      res.cache[i] := NEW(BDDTripleHash.Default).init(64)
    END; 
    EVAL BDDTripleHash.Put(res.tab, Pair { res.l, res.r }, (res));
    INC(nextId);
    RETURN res
  END New;

PROCEDURE Format(x : T; symtab : REFANY := NIL) : TEXT =
  VAR nm : TEXT;
  BEGIN
    IF symtab # NIL AND NARROW(symtab, BDDTextTbl.T).get(x, nm) THEN
      RETURN nm
    END;
    IF x = true THEN RETURN "TRUE"
    ELSIF x = false THEN RETURN "FALSE"
    END;
       
    IF x.name # NIL THEN RETURN x.name END;

    RETURN Fmt.Int(x.root.id) & " && (" & Format(x.l) & ") || (" & Format(x.r) &
           ") && ~" & Fmt.Int(x.root.id)
  END Format;

  use option @M3stackdump to get a stack trace
Abort


I'm happy to share the whole package if anyone with a better clue than me would like a look.

In fact:

http://rover.gcapltd.com/~mika/bdd.tgz

It should build without anything outside of standard cm3. 

     Mika


From mika at async.caltech.edu  Sun Jan 19 23:01:23 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 14:01:23 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119213649.403AE1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
Message-ID: <20140119220123.0D1971A208F@async.async.caltech.edu>

As in the ISTYPE/TYPECASE example, the following works:


PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(b)
    END;
  END Z;

(244)rover:~/refany/src>../FreeBSD4/refany
a


This does NOT work:

PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(x)
    END;
  END Z;


(242)rover:~/refany/src>../FreeBSD4/refany


***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 22
***

  use option @M3stackdump to get a stack trace
Abort


Same result PM3 / CM3.

     Mika


From jay.krell at cornell.edu  Sun Jan 19 23:58:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 22:58:27 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>,
	<DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
Message-ID: <COL130-W23436BD09BE1F20C442D6FE6BA0@phx.gbl>

1) portability; We currently have separate code, roughly, for MacOSX, FreeBSD, OpenBSD, NT, and the other Posixes.
MacOSX doesn't have Posix semaphores, for example.
This would give us one codebase here.
Only IA64 would likely remain an outlier, because stack is not merely the address of one local to another, but includes also the grow-up stack for the register saves.



2) It'd workaround the wow64 problem.


?- Jay

________________________________
> Subject: Re: [M3devel] Cooperative suspend? 
> From: dragisha at m3w.org 
> Date: Sun, 19 Jan 2014 15:28:18 +0100 
> CC: m3devel at elegosoft.com 
> To: jay.krell at cornell.edu 
> 
> Why would we do this, and not signalling, as in POSIX implementation? 
> 
> Why try to make something synchronous when we have working asynchronous 
> implementation? 
> 
> Why would it be better than sync one? 
> 
> On 19 Jan 2014, at 13:00, Jay K 
> <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote: 
> 
> There is a list of threads maintained by the m3 runtime. 
> Not by enumerating OS threads. 
> The thread structure contains in it perhaps a boolean or enum as to 
> suspend requested, suspending, suspended, resuming, running, etc. 
> 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:13:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:13:24 +0000
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>,
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <COL130-W156EFA4699196726508DB4E6BA0@phx.gbl>

"not worried" for native windows.
Still worried about wow64.


The answer for native is in Rotor/sscli and I have a program to show it working/not-working.
Specifically SuspendThread needs to be followed by GetThreadContext to ensure the suspend has completed.
If we don't do that, we must. If we already do, we are ok. I'll look later.
wow64 is still a problem. I'm still looking into it and thinking about it. But we do have native AMD64_NT
so just saying wow64 doesn't work might be the answer. I know people will protest that it works, but doesn't always.
I haven't figured out what rotor/sscli does.
Java uses cooperative suspend, which sidesteps this stuff.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Win32 SuspendThread?
> Date: Sun, 19 Jan 2014 12:05:48 -0800
> From: mika at async.caltech.edu
>
> "less worried" = "I think it works"?
>
> I think the only threading that works in CM3 is user threading...
>
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least). I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
>
> valgrind has a threading checker called "helgrind"...
>
> Mika
>
> Jay writes:
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/plain;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>>w up further.
>>
>> - Jay
>>
>>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>>
>>
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>>h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>>ow64.=20
>>> Anyone else please confirm:=20
>>> 1) my expectations -- it should never print anything
>>> 2) their importance -- garbage collector depends on it =20
>>> 3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>> while (1)
>>> InterlockedIncrement(&value);
>>> return 0;
>>> }
>>> int __cdecl main()
>>> {
>>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>> UINT i =3D 0;
>>> while (1)
>>> {
>>> i +=3D 1;
>>> if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>> {
>>> printf("suspend failed %X\n", GetLastError());
>>> Sleep(1);
>>> continue;
>>> }
>>> volatile long a =3D value;=20
>>> volatile long b =3D value;
>>> if (a !=3D b)
>>> {
>>> printf("%d %d %d %d\n", i, a, b, b - a);
>>> }
>>> ResumeThread(thread);
>>> }
>>> }
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/html;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>><blockquote type=3D"cite"><div>
>>
>><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>>
>><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>><style>
>>
>>
>><div class=3D"WordSection1">
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>>:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>>shes for me on both platforms.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>>r new test program yet.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>>p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><div>
>><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>>in 0in">
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>><b>On Behalf Of </b>Jay K<br>
>><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>><b>To:</b> m3devel<br>
>><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>></div>
>></div>
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>><div>
>><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>>in-bottom:12.0pt;margin-left:.5in">
>><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>>;This program also doesn't behave as expected, native, nothing to do with wo=
>>w64.
>><br>
>>&nbsp;Anyone else please confirm: <br>
>>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>>sp; <br>
>>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>>&nbsp;I am following up further. <br>
>>&nbsp;Maybe we should get cooperative suspend really going? <br>
>>Thank you.<br>
>>&nbsp;- Jay <br>
>>&nbsp;<br>
>>#include &lt;stdio.h&gt;<br>
>>#include &lt;windows.h&gt;<br>
>>volatile long value;<br>
>>unsigned long __stdcall Thread(PVOID parameter)<br>
>>{<br>
>>&nbsp; while (1)<br>
>>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>>&nbsp; return 0;<br>
>>}<br>
>>int __cdecl main()<br>
>>{<br>
>>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>>&nbsp; UINT i =3D 0;<br>
>>&nbsp; while (1)<br>
>>&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>>;<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>>&nbsp; }<br>
>>}<o:p></o:p></span></p>
>></div>
>></div>
>>
>>
>></div></blockquote></body></html>=
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:15:02 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:15:02 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <20140119201308.0A4B41A208F@async.async.caltech.edu>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
Message-ID: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>

The only problem I remember is an alignment matter using msvcrt.dll on AMD64_NT.
I'll look into it, and the old emails.
I've used the C backend on many platforms now.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com; adacore at marino.st
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> Date: Sun, 19 Jan 2014 12:13:08 -0800
> From: mika at async.caltech.edu
>
>
> There were some bugs we never got around to addressing with the C backend,
> weren't there? A crash that didn't happen with native backend.
>
> I've been using the "native" at more or less the head on very recent
> Debian and the Raspberry Pi for a while now with no problems to speak of.
> (As long as you only use user threads... but that's another story, I think.)
>
> Mika
>
> Jay K writes:
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/plain; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>>I understand your reluctance.
>>
>>
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources?
>>
>>
>>Yes=2C and to make sure the build process works for you.
>>
>>
>>> Shouldn't FreeBSD use the C backend too?
>>
>>
>>Every target should. I haven't convinced everyone yet.
>>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>>which are working=2C using the C backend.
>>There are downsides to the C backend:
>> - it is slower to compiler =20
>> - debugging is degraded on systems that have m3gdb =20
>> (debugging is much better on systems w/o m3gdb)=20
>>
>>
>>And there are advantages not yet implemented=2C in particular=2C efficient =
>>exception handling by generating C++.
>>
>>
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>
>>
>>Your patches should apply fairly cleanly to any new checkout.
>>I am willing to through them and apply them to CVS if you want -- depends o=
>>n if you are ok with my stealing your credit=2C vs. you want to commit them=
>> yourselves.
>>I already skimmed them and they all look easy.
>>DragonflyBSD is basically FreeBSD by another name.
>>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>>orts.
>>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>>Sure=2C they might be implemented differently=2C but they are highly source=
>> compatible and every highly object code compatible.
>>
>>
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>
>>Agreed=2C we are both missing something simple.
>>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>>3.
>>If you want to press on with the current approach and ignore my advise=2C i=
>>t should be possible to step through all this.
>>To start=2C break on Target__Init and see if it returns true or false.
>>
>>
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>
>>
>>Ugh. I saw something about that in your diffs. Definitely we intend to link=
>> statically.
>>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>>ource.
>>They are used for actually very little and aren't worth it.
>>
>>
>> - Jay
>>
>>
>>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>>> From: adacore at marino.st
>>> To: jay.krell at cornell.edu
>>> CC: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>>=20
>>> On 1/19/2014 13:17=2C Jay K wrote:
>>>> Hey..I like that that does capture some of My Way.=20
>>>> Here is my independent off the cuff write up though:
>>>> I know the ports system is nice.
>>>> I know we really need to be in it for many operating systems.
>>>> I suspect your problem is that you didn't "upgrade" to apply you
>>>> changes at the right time.
>>>=20
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>>=20
>>>=20
>>>> Do any of these work for you:
>>>> https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>>=20
>>> I'm not clear to which set you are referring.
>>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>>> superior to the release binaries because they link gmp and mpfr
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>>=20
>>> Then there are the source distributions=2C but they are from 2012 so they
>>> aren't exactly fresh either.
>>>=20
>>>=20
>>>> Anyway=2C please try this:=20
>>>> Ignore DragonFly at first.=20
>>>> Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>>>> anything.=20
>>>> Such as from ports.=20
>>>> Make a backup=2C e.g. of /usr/local/cm3.=20
>>>> My instructions are unfortunately destructive of the existing install=
>>.=20
>>>> Checkout the full current CVS repository.=20
>>>> In that repository=2C run scripts/python/upgrade.py.=20
>>>> If that doesn't work=2C stop=2C and we'll help.=20
>>>=20
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources? If so=2C I'd just modify the port to do that now that we have
>>> working binary bootstraps. The main reason I'm reluctant is that it
>>> took over a week to make the cm3 port=2C I've never had so much difficult=
>>y
>>> before. (I've also spent another week on the DragonFly port). So I'm
>>> "resisting" because of how much work it already took. That's why I was
>>> trying to port DragonFly=2C THEN update to latest rather than update to
>>> latest and then port DragonFly. (in other words=2C going forward seems a
>>> short path than backtracking)
>>>=20
>>>=20
>>>=20
>>>> If that does work=2C apply your diffs. And upgrade.py again. You can =
>>say
>>>> "upgrade.py skipgcc" here.=20
>>>> Then boot1.py c amd64_dragonflybsd =20
>>>> Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>>re
>>>> on the command line.=20
>>>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>>=2C
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>>=20
>>>=20
>>>> If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>>gz=20
>>>> Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>>e.=20
>>>> If that works=2C you have a native cm3.=20
>>>> Run it. It should say "can't find cm3.cfg". =20
>>>> Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>>>> cp cm3 /usr/local/cm3/bin =20
>>>> export PATH=3D/usr/local/cm3/bin:$PATH
>>>> full CVS checkout =20
>>>> cd scripts/python =3B ./boot2.sh=20
>>>> =20
>>>> You will need to have made small changes to pylib.py for this to work=
>>.
>>>> (And your config file should say to use the C backend=2C like Darwin =
>>and
>>>> ARM_LINUX do.)
>>>=20
>>>=20
>>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>>ong
>>> time. Shouldn't FreeBSD use the C backend too?
>>>=20
>>> John
>> =
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/html; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>><html>
>><head>
>><style></style></head>
>><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>>h are working=2C using the C backend.<br>There are downsides to the C backe=
>>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
>> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>>et implemented=2C in particular=2C efficient exception handling by generati=
>>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>>=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>>=3B <br>&gt=3B I assume you mean after the system understands this target. =
>> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'=
>>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
>> am willing to through them and apply them to CVS if you want -- depends on=
>> if you are ok with my stealing your credit=2C vs. you want to commit them =
>>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>>e highly source compatible and every highly object code compatible.<br><br>=
>><br>&gt=3B I must be misunderstanding what you mean by "upgrade". It's the=
>><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>>r>If you want to press on with the current approach and ignore my advise=2C=
>> it should be possible to step through all this.<br>To start=2C break on Ta=
>>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
>><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
>> to link statically.<br>I actually went to the trouble of removing use of g=
>>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
>> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>>dependent off the cuff write up though:<br>&gt=3B &gt=3B I know the ports =
>>system is nice.<br>&gt=3B &gt=3B I know we really need to be in it for man=
>>y operating systems.<br>&gt=3B &gt=3B I suspect your problem is that you d=
>>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade". =
>>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
>> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Do any of thes=
>>e work for you:<br>&gt=3B &gt=3B https://modula3.elegosoft.com/cm3/snaps/s=
>>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
>> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Anyway=2C plea=
>>se try this: <br>&gt=3B &gt=3B Ignore DragonFly at first. <br>&gt=3B &gt=
>>=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B Such as from ports. <br>&gt=3B =
>>&gt=3B Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B My in=
>>structions are unfortunately destructive of the existing install. <br>&gt=
>>=3B &gt=3B Checkout the full current CVS repository. <br>&gt=3B &gt=3B =
>>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B If=
>> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>>? If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>>working binary bootstraps. The main reason I'm reluctant is that it<br>&gt=
>>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>>lty<br>&gt=3B before. (I've also spent another week on the DragonFly port)=
>>. So I'm<br>&gt=3B "resisting" because of how much work it already took. =
>>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>>t rather than update to<br>&gt=3B latest and then port DragonFly. (in othe=
>>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B If that does work=2C appl=
>>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>>y skipgcc" here. <br>&gt=3B &gt=3B Then boot1.py c amd64_dragonflybsd <=
>>br>&gt=3B &gt=3B Make sure you say "c" and the target "amd64_dragonflyb=
>>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B =
>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B for Linux=2C=
>> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>>me you mean after the system understands this target. There will<br>&gt=3B=
>> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>>FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to<br>&gt=3B =
>>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>>r>&gt=3B &gt=3B If this works=2C it will give you=2C roughly amd64_dragon=
>>flybsd*.tar.gz <br>&gt=3B &gt=3B Copy that to your Dragonfly system=2C ex=
>>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B If that works=2C you ha=
>>ve a native cm3. <br>&gt=3B &gt=3B Run it. It should say "can't find cm3.=
>>cfg". <br>&gt=3B &gt=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
>><br>&gt=3B &gt=3B cp cm3 /usr/local/cm3/bin <br>&gt=3B &gt=3B export P=
>>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B full CVS checkout <br>&g=
>>t=3B &gt=3B cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B <br>&gt=
>>=3B &gt=3B You will need to have made small changes to pylib.py for this =
>>to work.<br>&gt=3B &gt=3B (And your config file should say to use the C b=
>>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>>hat takes a long<br>&gt=3B time. Shouldn't FreeBSD use the C backend too?<=
>>br>&gt=3B <br>&gt=3B John<br></div> </div></body>
>></html>=
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- 		 	   		  

From mika at async.caltech.edu  Mon Jan 20 00:39:38 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 15:39:38 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
	<COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
Message-ID: <20140119233938.9C5A31A208F@async.async.caltech.edu>

Jay K writes:
>The only problem I remember is an alignment matter using msvcrt.dll on AMD6=
>4_NT.=0A=
>I'll look into it=2C and the old emails.=0A=
>I've used the C backend on many platforms now.=0A=

This stuff doesn't happen with the native backend:

> 2. I can't get anything with Trestle working. I'm running in a VNC
> session on a remote machine. Not sure this has anything to do with the
> compiler or even the ARM. Has anyone seen it before (on any system)?
>
> (gdb) cont
> Continuing.
> [New Thread 0x42276430 (LWP 24451)]
> [Thread 0x42276430 (LWP 24451) exited]
> [New Thread 0x424c6430 (LWP 24454)]
>
>
> ***
> *** runtime error:
> *** Exception "ZChildVBT.BadPercentage" not in RAISES list
> *** file "../src/lego/ZChildVBT.m3", line 61
> ***
>
>
> Program received signal SIGABRT, Aborted.
>
> backtrace:
>
> (gdb) where
> #0 ZChildVBT__Init (z_L_120=0x422bbf70 "H\357K", ch_L_121=0x422bdb84 "\260\326K", h_L_122=0, v_L_123=1.75, loc_L_124=4 '\004', type_L_125=1 '\001', shaper_
L_126=0x41a32b54, open_L_127=0 '\000') at ../src/lego/ZCh
> ildVBT.m3:61
> #1 0x40637518 in FormsVBT__pZChild (cl_L_1042=0x42280444, list_L_1043=0x424c54c4, s_L_1044=0x424c55dc) at ../src/FormsVBT.m3:2593
> #2 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a45994 "", state_L_303=0x424c55dc) at ../src/FormsVBT.m3:250
> #3 0x40648b3c in FormsVBT__AddChildren (cl_L_1229=0x42280444, v_L_1230=0x422b0634 " \365K", list_L_1231=0x41a48504, state_L_1232=0x424c55dc) at ../src/FormsVBT.m3:3671
> #4 0x40634138 in FormsVBT__pZSplit (cl_L_999=0x42280444, list_L_1000=0x424c56bc, s_L_1001=0x424c57c4) at ../src/FormsVBT.m3:2454
> #5 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a40ec0 "", state_L_303=0x424c57c4) at ../src/FormsVBT.m3:250
> #6 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c57c4) at ../src/FormsVBT.m3:3642
> #7 0x406129c0 in FormsVBT__pShape (cl_L_496=0x42280444, list_L_497=0x424c58a4, s_L_498=0x424c59a4) at ../src/FormsVBT.m3:948
> #8 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a402a4 "", state_L_303=0x424c59a4) at ../src/FormsVBT.m3:250
> #9 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c59a4) at ../src/FormsVBT.m3:3642
> #10 0x4062ca10 in FormsVBT__pScale (cl_L_905=0x42280444, list_L_906=0x424c5a84, s_L_907=0x42280458) at ../src/FormsVBT.m3:2165
> #11 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a4006c "", state_L_303=0x42280458) at ../src/FormsVBT.m3:250
> #12 0x40606dec in FormsVBT__Apply (cl_L_293=0x42280444) at ../src/FormsVBT.m3:84
> #13 0x40d27bd4 in ThreadPThread__RunThread (me_L_231=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:449
> #14 0x40d27590 in ThreadPThread__ThreadBase (param_L_229=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:422
> #15 0x41850bfc in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
> #16 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> #17 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> (gdb) cont
> Continuing.
>



From jay.krell at cornell.edu  Mon Jan 20 01:06:27 2014
From: jay.krell at cornell.edu (Jay)
Date: Sun, 19 Jan 2014 16:06:27 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <FC1A38F9-1A18-4DD1-99BB-9FB53669C761@gmail.com>

Cooperative suspend will also fix this signal allocation vs. valgrind problem. Can also likely pick a different one...

 - Jay

On Jan 19, 2014, at 12:05 PM, <mika at async.caltech.edu> wrote:

> "less worried" = "I think it works"?
> 
> I think the only threading that works in CM3 is user threading...
> 
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least).  I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
> 
> valgrind has a threading checker called "helgrind"...
> 
>     Mika
> 
> Jay writes:
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/plain;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>> w up further.
>> 
>> - Jay
>> 
>> On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>> 
>> 
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>> h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>> ow64.=20
>>> Anyone else please confirm:=20
>>>   1) my expectations -- it should never print anything
>>>   2) their importance  -- garbage collector depends on it =20
>>>   3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>>  while (1)
>>>   InterlockedIncrement(&value);
>>>  return 0;
>>> }
>>> int __cdecl main()
>>> {
>>>  HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>>  UINT i =3D 0;
>>>  while (1)
>>>  {
>>>    i +=3D 1;
>>>    if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>>    {
>>>      printf("suspend failed %X\n", GetLastError());
>>>      Sleep(1);
>>>      continue;
>>>    }
>>>    volatile long a =3D value;=20
>>>    volatile long b =3D value;
>>>    if (a !=3D b)
>>>    {
>>>     printf("%d %d %d %d\n", i, a, b, b - a);
>>>    }
>>>    ResumeThread(thread);
>>>  }
>>> }
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/html;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>> utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>> ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>> div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>> mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>> <blockquote type=3D"cite"><div>
>> 
>> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>> 
>> <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>> <style><!--
>> /* Font Definitions */
>> @font-face
>>    {font-family:Calibri;
>>    panose-1:2 15 5 2 2 2 4 3 2 4;}
>> @font-face
>>    {font-family:Tahoma;
>>    panose-1:2 11 6 4 3 5 4 4 2 4;}
>> /* Style Definitions */
>> p.MsoNormal, li.MsoNormal, div.MsoNormal
>>    {margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> a:link, span.MsoHyperlink
>>    {mso-style-priority:99;
>>    color:blue;
>>    text-decoration:underline;}
>> a:visited, span.MsoHyperlinkFollowed
>>    {mso-style-priority:99;
>>    color:purple;
>>    text-decoration:underline;}
>> p
>>    {mso-style-priority:99;
>>    mso-margin-top-alt:auto;
>>    margin-right:0in;
>>    mso-margin-bottom-alt:auto;
>>    margin-left:0in;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>>    {mso-style-priority:99;
>>    mso-style-link:"Balloon Text Char";
>>    margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:8.0pt;
>>    font-family:"Tahoma","sans-serif";}
>> span.EmailStyle18
>>    {mso-style-type:personal-reply;
>>    font-family:"Calibri","sans-serif";
>>    color:#1F497D;}
>> span.BalloonTextChar
>>    {mso-style-name:"Balloon Text Char";
>>    mso-style-priority:99;
>>    mso-style-link:"Balloon Text";
>>    font-family:"Tahoma","sans-serif";}
>> .MsoChpDefault
>>    {mso-style-type:export-only;
>>    font-size:10.0pt;}
>> @page WordSection1
>>    {size:8.5in 11.0in;
>>    margin:1.0in 1.0in 1.0in 1.0in;}
>> div.WordSection1
>>    {page:WordSection1;}
>> --></style><!--[if gte mso 9]><xml>
>> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
>> </xml><![endif]--><!--[if gte mso 9]><xml>
>> <o:shapelayout v:ext=3D"edit">
>> <o:idmap v:ext=3D"edit" data=3D"1" />
>> </o:shapelayout></xml><![endif]-->
>> 
>> 
>> <div class=3D"WordSection1">
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>> bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>> :p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>> shes for me on both platforms.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>> r new test program yet.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>> p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <div>
>> <div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>> in 0in">
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>> e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>> </b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>> s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>> a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>> <b>On Behalf Of </b>Jay K<br>
>> <b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>> <b>To:</b> m3devel<br>
>> <b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>> </div>
>> </div>
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>> <div>
>> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>> in-bottom:12.0pt;margin-left:.5in">
>> <span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>> ;This program also doesn't behave as expected, native, nothing to do with wo=
>> w64.
>> <br>
>> &nbsp;Anyone else please confirm: <br>
>> &nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>> &nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>> sp; <br>
>> &nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>> &nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>> &nbsp;I am following up further. <br>
>> &nbsp;Maybe we should get cooperative suspend really going? <br>
>> Thank you.<br>
>> &nbsp;- Jay <br>
>> &nbsp;<br>
>> #include &lt;stdio.h&gt;<br>
>> #include &lt;windows.h&gt;<br>
>> volatile long value;<br>
>> unsigned long __stdcall Thread(PVOID parameter)<br>
>> {<br>
>> &nbsp; while (1)<br>
>> &nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>> &nbsp; return 0;<br>
>> }<br>
>> int __cdecl main()<br>
>> {<br>
>> &nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>> &nbsp; UINT i =3D 0;<br>
>> &nbsp; while (1)<br>
>> &nbsp; {<br>
>> &nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>> &nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>> ;<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>> &nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>> &nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>> &nbsp; }<br>
>> }<o:p></o:p></span></p>
>> </div>
>> </div>
>> 
>> 
>> </div></blockquote></body></html>=
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From rodney_bates at lcwb.coop  Mon Jan 20 17:00:13 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 10:00:13 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD480D.3000101@lcwb.coop>

What does this do?

PROCEDURE Z(VAR x : REFANY) =
   VAR b: BDD.T;
   BEGIN
     b := x;
     x := BDD.Format(b)
   END Z;

The common property of the failing cases seems to be the necessity
of a runtime narrow check when passing x to BDD.Format.

So far, I haven't been able to reproduce the failure on AMD64_LINUX,
with even more irrelevant (one would think) stuff pared out.  I moved
T, Root, and Format into main, removed all but one fields of T and Root,
replaced BDD.New by NEW(Root) and removed Symtab and most of the body
from Format.

Looking at your code, I agree it should work.


On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From rodney_bates at lcwb.coop  Mon Jan 20 18:02:40 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 11:02:40 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD56B0.1050007@lcwb.coop>

Lots more experiments, still no reproduction of the problem on AMD64_LINUX.
The 5 cases in the program below all work.  But earlier versions of this
test program, with bugs, (failure to repeat x:=y before every Z* call)
suggested a hypothesis.

For x := BDD.Format(x)

Perhaps the compiler does something to the LHS x (zeros it out?) before
narrowing and passing the x that is an actual parameter to Format.

If so, it is an implementation bug, as the language says (2.3.1):

   (for v := e)

   "The order of evaluation of v and e is undefined, but e will be evaluated
    before v is updated."

_____________________________________________________________________________________

MODULE Main;

TYPE T = OBJECT l : T; END;

TYPE Root = T OBJECT id : CARDINAL; END;

PROCEDURE Format(x : T) : TEXT =
   BEGIN
     RETURN "Format"
   END Format;

PROCEDURE P() : REFANY =
   BEGIN
     VAR x, y : REFANY;
     BEGIN
       y := NEW ( Root );
       x := y;
       Z1(x);
       x := y;
       Z2(x);
       x := y;
       Z3(x);
       x := y;
       Z4(x);
       x := y;
       Z5(x);
       RETURN x
     END
   END P;

PROCEDURE Z1(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T (b) => x := Format(b)
     END;
   END Z1;

PROCEDURE Z2(VAR x : REFANY) =
   VAR b : T;
   BEGIN
     b := x;
     x := Format(b)
   END Z2;

PROCEDURE Z3(VAR x : REFANY) =
   BEGIN
     x := Format(x)
   END Z3;

PROCEDURE Z4(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T => x := Format (x)
     END;
   END Z4;

PROCEDURE Z5(VAR x : REFANY) =
   BEGIN
     IF ISTYPE (x, T) THEN
       x := Format (x)
     END;
   END Z5;

BEGIN
   EVAL P()
END Main.

______________________________________________________________________________________

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Tue Jan 21 05:18:26 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Mon, 20 Jan 2014 20:18:26 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DD480D.3000101@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
Message-ID: <20140121041826.4D95D1A2080@async.async.caltech.edu>


Were you able to get it to fail on any of your computers?

Your Z fails for me---CM3 and PM3 both.

(579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../src/Main.m3", line 30
***

Abort
(580)truffles:~/bsd/refany/src>cat Mai
Main.m3~  Main.m3   
(580)truffles:~/bsd/refany/src>cat -n Main.m3
     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  (*
    20  PROCEDURE Z(VAR x : REFANY) =
    21    BEGIN
    22      TYPECASE x OF
    23        BDD.T(b) => x := BDD.Format(b)
    24      END;
    25    END Z;
    26  *)
    27  PROCEDURE Z(VAR x : REFANY) =
    28     VAR b: BDD.T;
    29     BEGIN
    30       b := x;
    31       x := BDD.Format(b)
    32     END Z;
    33
    34
    35
    36  BEGIN
    37    Q()
    38  END Main.
(581)truffles:~/bsd/refany/src>
"Rodney M. Bates" writes:
>What does this do?
>
>PROCEDURE Z(VAR x : REFANY) =
>   VAR b: BDD.T;
>   BEGIN
>     b := x;
>     x := BDD.Format(b)
>   END Z;
>
>The common property of the failing cases seems to be the necessity
>of a runtime narrow check when passing x to BDD.Format.
>
>So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>with even more irrelevant (one would think) stuff pared out.  I moved
>T, Root, and Format into main, removed all but one fields of T and Root,
>replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>from Format.
>
>Looking at your code, I agree it should work.
>
>
>On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>> As in the ISTYPE/TYPECASE example, the following works:
>>
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(b)
>>      END;
>>    END Z;
>>
>> (244)rover:~/refany/src>../FreeBSD4/refany
>> a
>>
>>
>> This does NOT work:
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(x)
>>      END;
>>    END Z;
>>
>>
>> (242)rover:~/refany/src>../FreeBSD4/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    NARROW failed
>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>> ***
>>
>>    use option @M3stackdump to get a stack trace
>> Abort
>>
>>
>> Same result PM3 / CM3.
>>
>>       Mika
>>


From rodney_bates at lcwb.coop  Tue Jan 21 17:34:57 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 10:34:57 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEA1B1.2070808@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?

No, the only times I got narrow failures were due to errors
in my trial cases.

I have only tried AMD64_LINUX, but should be able to get this onto a
LINUXLIBC6 system.

>
> Your Z fails for me---CM3 and PM3 both.
>
> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>
>
> ***
> *** runtime error:
> ***    An explicit or implicit NARROW() operation failed.
> ***    file "../src/Main.m3", line 30
> ***

Well that torpedos both my theories.

But it is hard to imagine that bad code generated for an implicit narrow
on an assignment could ever have gone unnoticed so long.  I thought it
might be more believable if it only happened when narrowing an actual
parameter before passing it.

>
> Abort
> (580)truffles:~/bsd/refany/src>cat Mai
> Main.m3~  Main.m3
> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>       1  MODULE Main;
>       2  IMPORT BDD, IO;
>       3
>       4  PROCEDURE P() : REFANY =
>       5    BEGIN
>       6      VAR x : REFANY;
>       7      BEGIN
>       8        x := BDD.New("a");
>       9        Z(x);
>      10        RETURN x
>      11      END
>      12    END P;
>      13
>      14  PROCEDURE Q() =
>      15    BEGIN
>      16      IO.Put(P() & "\n")
>      17    END Q;
>      18
>      19  (*
>      20  PROCEDURE Z(VAR x : REFANY) =
>      21    BEGIN
>      22      TYPECASE x OF
>      23        BDD.T(b) => x := BDD.Format(b)
>      24      END;
>      25    END Z;
>      26  *)
>      27  PROCEDURE Z(VAR x : REFANY) =
>      28     VAR b: BDD.T;
>      29     BEGIN
>      30       b := x;
>      31       x := BDD.Format(b)
>      32     END Z;
>      33
>      34
>      35
>      36  BEGIN
>      37    Q()
>      38  END Main.
> (581)truffles:~/bsd/refany/src>
> "Rodney M. Bates" writes:
>> What does this do?
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    VAR b: BDD.T;
>>    BEGIN
>>      b := x;
>>      x := BDD.Format(b)
>>    END Z;
>>
>> The common property of the failing cases seems to be the necessity
>> of a runtime narrow check when passing x to BDD.Format.
>>
>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>> with even more irrelevant (one would think) stuff pared out.  I moved
>> T, Root, and Format into main, removed all but one fields of T and Root,
>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>from Format.
>>
>> Looking at your code, I agree it should work.
>>
>>
>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>> As in the ISTYPE/TYPECASE example, the following works:
>>>
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(b)
>>>       END;
>>>     END Z;
>>>
>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>> a
>>>
>>>
>>> This does NOT work:
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(x)
>>>       END;
>>>     END Z;
>>>
>>>
>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    NARROW failed
>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>> ***
>>>
>>>     use option @M3stackdump to get a stack trace
>>> Abort
>>>
>>>
>>> Same result PM3 / CM3.
>>>
>>>        Mika
>>>
>



From mika at async.caltech.edu  Tue Jan 21 18:11:20 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 09:11:20 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEA1B1.2070808@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
Message-ID: <20140121171120.F21FF1A2080@async.async.caltech.edu>

Did you try using my bdd.tgz ?

I am getting this error on: 

ancient PM3 on FreeBSD4

head CM3 on AMD64_LINUX

few months old CM3 on ARM_LINUX (Raspberry Pi)

I doubt you will get it on LINUXLIBC6 if you don't get it on the others.

I already tried rearranging the files (the implementation and interfaces
are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
right now).  With no effect.  I don't know what it is about my code that's
tripping up the compiler.

It's also a "decent" BDD package so someone (me I guess, but anyone
else who's interested is welcome to) might want to add it to CM3... :-)
Kind of a pain if you can't print out the debugging info though!
(Actually the workarounds are OK.)

    Mika

"Rodney M. Bates" writes:
>
>
>On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>> Were you able to get it to fail on any of your computers?
>
>No, the only times I got narrow failures were due to errors
>in my trial cases.
>
>I have only tried AMD64_LINUX, but should be able to get this onto a
>LINUXLIBC6 system.
>
>>
>> Your Z fails for me---CM3 and PM3 both.
>>
>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    An explicit or implicit NARROW() operation failed.
>> ***    file "../src/Main.m3", line 30
>> ***
>
>Well that torpedos both my theories.
>
>But it is hard to imagine that bad code generated for an implicit narrow
>on an assignment could ever have gone unnoticed so long.  I thought it
>might be more believable if it only happened when narrowing an actual
>parameter before passing it.
>
>>
>> Abort
>> (580)truffles:~/bsd/refany/src>cat Mai
>> Main.m3~  Main.m3
>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>       1  MODULE Main;
>>       2  IMPORT BDD, IO;
>>       3
>>       4  PROCEDURE P() : REFANY =
>>       5    BEGIN
>>       6      VAR x : REFANY;
>>       7      BEGIN
>>       8        x := BDD.New("a");
>>       9        Z(x);
>>      10        RETURN x
>>      11      END
>>      12    END P;
>>      13
>>      14  PROCEDURE Q() =
>>      15    BEGIN
>>      16      IO.Put(P() & "\n")
>>      17    END Q;
>>      18
>>      19  (*
>>      20  PROCEDURE Z(VAR x : REFANY) =
>>      21    BEGIN
>>      22      TYPECASE x OF
>>      23        BDD.T(b) => x := BDD.Format(b)
>>      24      END;
>>      25    END Z;
>>      26  *)
>>      27  PROCEDURE Z(VAR x : REFANY) =
>>      28     VAR b: BDD.T;
>>      29     BEGIN
>>      30       b := x;
>>      31       x := BDD.Format(b)
>>      32     END Z;
>>      33
>>      34
>>      35
>>      36  BEGIN
>>      37    Q()
>>      38  END Main.
>> (581)truffles:~/bsd/refany/src>
>> "Rodney M. Bates" writes:
>>> What does this do?
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>    VAR b: BDD.T;
>>>    BEGIN
>>>      b := x;
>>>      x := BDD.Format(b)
>>>    END Z;
>>>
>>> The common property of the failing cases seems to be the necessity
>>> of a runtime narrow check when passing x to BDD.Format.
>>>
>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>from Format.
>>>
>>> Looking at your code, I agree it should work.
>>>
>>>
>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(b)
>>>>       END;
>>>>     END Z;
>>>>
>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>> a
>>>>
>>>>
>>>> This does NOT work:
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(x)
>>>>       END;
>>>>     END Z;
>>>>
>>>>
>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    NARROW failed
>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>> ***
>>>>
>>>>     use option @M3stackdump to get a stack trace
>>>> Abort
>>>>
>>>>
>>>> Same result PM3 / CM3.
>>>>
>>>>        Mika
>>>>
>>


From rodney_bates at lcwb.coop  Tue Jan 21 18:23:41 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 11:23:41 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEAD1D.8010602@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?
>

My test case with 5 versions of Z, and your complete one that you posted
(bdd.tgz) both silently complete with no error messages on LINUXLIBC6,
using the head compiler and libraries.

bdd.tgz also works fine on AMD64_LINUX.

On yours, I built and shipped in bdd/src, then built in bdd/test/src
and ran LINUXLIBC6/bddtest, which I presume was the right procedure.

How about compiling a couple of failing Z versions on a machine where
they fail, with -keep, then posting the .mc, .ms, and .mo files and corresponding
source (only for the Zs is probably enough).


>>>        Mika
>>>
>



From rodney_bates at lcwb.coop  Tue Jan 21 19:02:09 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 12:02:09 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121171120.F21FF1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
Message-ID: <52DEB621.7080706@lcwb.coop>



On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
> Did you try using my bdd.tgz ?
>
> I am getting this error on:
>
> ancient PM3 on FreeBSD4
>
> head CM3 on AMD64_LINUX

Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
slightly different in size when I recompiled, but I think there is an explanation
for that.

I can't build bddtest with the release compiler, because it needs caltech_parser,
and that won't build in the release.  This combination of [non]failures seems
pretty strange.  I am also wondering about a problem with type checks in the
runtime.


>
> few months old CM3 on ARM_LINUX (Raspberry Pi)
>
> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>
> I already tried rearranging the files (the implementation and interfaces
> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
> right now).  With no effect.  I don't know what it is about my code that's
> tripping up the compiler.
>
> It's also a "decent" BDD package so someone (me I guess, but anyone
> else who's interested is welcome to) might want to add it to CM3... :-)
> Kind of a pain if you can't print out the debugging info though!
> (Actually the workarounds are OK.)
>
>      Mika
>
> "Rodney M. Bates" writes:
>>
>>
>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>> Were you able to get it to fail on any of your computers?
>>
>> No, the only times I got narrow failures were due to errors
>> in my trial cases.
>>
>> I have only tried AMD64_LINUX, but should be able to get this onto a
>> LINUXLIBC6 system.
>>
>>>
>>> Your Z fails for me---CM3 and PM3 both.
>>>
>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    An explicit or implicit NARROW() operation failed.
>>> ***    file "../src/Main.m3", line 30
>>> ***
>>
>> Well that torpedos both my theories.
>>
>> But it is hard to imagine that bad code generated for an implicit narrow
>> on an assignment could ever have gone unnoticed so long.  I thought it
>> might be more believable if it only happened when narrowing an actual
>> parameter before passing it.
>>
>>>
>>> Abort
>>> (580)truffles:~/bsd/refany/src>cat Mai
>>> Main.m3~  Main.m3
>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>        1  MODULE Main;
>>>        2  IMPORT BDD, IO;
>>>        3
>>>        4  PROCEDURE P() : REFANY =
>>>        5    BEGIN
>>>        6      VAR x : REFANY;
>>>        7      BEGIN
>>>        8        x := BDD.New("a");
>>>        9        Z(x);
>>>       10        RETURN x
>>>       11      END
>>>       12    END P;
>>>       13
>>>       14  PROCEDURE Q() =
>>>       15    BEGIN
>>>       16      IO.Put(P() & "\n")
>>>       17    END Q;
>>>       18
>>>       19  (*
>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>       21    BEGIN
>>>       22      TYPECASE x OF
>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>       24      END;
>>>       25    END Z;
>>>       26  *)
>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>       28     VAR b: BDD.T;
>>>       29     BEGIN
>>>       30       b := x;
>>>       31       x := BDD.Format(b)
>>>       32     END Z;
>>>       33
>>>       34
>>>       35
>>>       36  BEGIN
>>>       37    Q()
>>>       38  END Main.
>>> (581)truffles:~/bsd/refany/src>
>>> "Rodney M. Bates" writes:
>>>> What does this do?
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     VAR b: BDD.T;
>>>>     BEGIN
>>>>       b := x;
>>>>       x := BDD.Format(b)
>>>>     END Z;
>>>>
>>>> The common property of the failing cases seems to be the necessity
>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>
>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>> >from Format.
>>>>
>>>> Looking at your code, I agree it should work.
>>>>
>>>>
>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>> a
>>>>>
>>>>>
>>>>> This does NOT work:
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>>
>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>
>>>>>
>>>>> ***
>>>>> *** runtime error:
>>>>> ***    NARROW failed
>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>> ***
>>>>>
>>>>>      use option @M3stackdump to get a stack trace
>>>>> Abort
>>>>>
>>>>>
>>>>> Same result PM3 / CM3.
>>>>>
>>>>>         Mika
>>>>>
>>>
>



From mika at async.caltech.edu  Tue Jan 21 19:08:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 10:08:06 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEB621.7080706@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
Message-ID: <20140121180806.D75A61A208F@async.async.caltech.edu>

The only thing bddtest uses from caltech_parser is Debug.Out.  You can just take out those calls or replace them with IO.Put.

But again bddtest is not the program that fails... it's the "refany" program (now in a sibling directory to bddtest)

   Mika

"Rodney M. Bates" writes:
>
>
>On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>> Did you try using my bdd.tgz ?
>>
>> I am getting this error on:
>>
>> ancient PM3 on FreeBSD4
>>
>> head CM3 on AMD64_LINUX
>
>Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
>a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
>but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
>slightly different in size when I recompiled, but I think there is an explanation
>for that.
>
>I can't build bddtest with the release compiler, because it needs caltech_parser,
>and that won't build in the release.  This combination of [non]failures seems
>pretty strange.  I am also wondering about a problem with type checks in the
>runtime.
>
>
>>
>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>
>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>>
>> I already tried rearranging the files (the implementation and interfaces
>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>> right now).  With no effect.  I don't know what it is about my code that's
>> tripping up the compiler.
>>
>> It's also a "decent" BDD package so someone (me I guess, but anyone
>> else who's interested is welcome to) might want to add it to CM3... :-)
>> Kind of a pain if you can't print out the debugging info though!
>> (Actually the workarounds are OK.)
>>
>>      Mika
>>
>> "Rodney M. Bates" writes:
>>>
>>>
>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>> Were you able to get it to fail on any of your computers?
>>>
>>> No, the only times I got narrow failures were due to errors
>>> in my trial cases.
>>>
>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>> LINUXLIBC6 system.
>>>
>>>>
>>>> Your Z fails for me---CM3 and PM3 both.
>>>>
>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    An explicit or implicit NARROW() operation failed.
>>>> ***    file "../src/Main.m3", line 30
>>>> ***
>>>
>>> Well that torpedos both my theories.
>>>
>>> But it is hard to imagine that bad code generated for an implicit narrow
>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>> might be more believable if it only happened when narrowing an actual
>>> parameter before passing it.
>>>
>>>>
>>>> Abort
>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>> Main.m3~  Main.m3
>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>        1  MODULE Main;
>>>>        2  IMPORT BDD, IO;
>>>>        3
>>>>        4  PROCEDURE P() : REFANY =
>>>>        5    BEGIN
>>>>        6      VAR x : REFANY;
>>>>        7      BEGIN
>>>>        8        x := BDD.New("a");
>>>>        9        Z(x);
>>>>       10        RETURN x
>>>>       11      END
>>>>       12    END P;
>>>>       13
>>>>       14  PROCEDURE Q() =
>>>>       15    BEGIN
>>>>       16      IO.Put(P() & "\n")
>>>>       17    END Q;
>>>>       18
>>>>       19  (*
>>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>>       21    BEGIN
>>>>       22      TYPECASE x OF
>>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>>       24      END;
>>>>       25    END Z;
>>>>       26  *)
>>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>>       28     VAR b: BDD.T;
>>>>       29     BEGIN
>>>>       30       b := x;
>>>>       31       x := BDD.Format(b)
>>>>       32     END Z;
>>>>       33
>>>>       34
>>>>       35
>>>>       36  BEGIN
>>>>       37    Q()
>>>>       38  END Main.
>>>> (581)truffles:~/bsd/refany/src>
>>>> "Rodney M. Bates" writes:
>>>>> What does this do?
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>     VAR b: BDD.T;
>>>>>     BEGIN
>>>>>       b := x;
>>>>>       x := BDD.Format(b)
>>>>>     END Z;
>>>>>
>>>>> The common property of the failing cases seems to be the necessity
>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>
>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>> >from Format.
>>>>>
>>>>> Looking at your code, I agree it should work.
>>>>>
>>>>>
>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>> a
>>>>>>
>>>>>>
>>>>>> This does NOT work:
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>>
>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>
>>>>>>
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    NARROW failed
>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>> ***
>>>>>>
>>>>>>      use option @M3stackdump to get a stack trace
>>>>>> Abort
>>>>>>
>>>>>>
>>>>>> Same result PM3 / CM3.
>>>>>>
>>>>>>         Mika
>>>>>>
>>>>
>>


From mika at async.caltech.edu  Tue Jan 21 20:23:28 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 11:23:28 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
	<20140121180806.D75A61A208F@async.async.caltech.edu>
	<07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
Message-ID: <20140121192328.92DDB1A208F@async.async.caltech.edu>

How do you get them into text form?  They look binary...

I copied the whole directory bdd (and all subdirs) to 

http://rover.gcapltd.com/~mika/bdd/

Antony Hosking writes:
>Please post .io / .mo in text form.
>
>Sent from my iPad
>
>> On Jan 21, 2014, at 1:08 PM, mika at async.caltech.edu wrote:
>>=20
>> The only thing bddtest uses from caltech_parser is Debug.Out.  You can jus=
>t take out those calls or replace them with IO.Put.
>>=20
>> But again bddtest is not the program that fails... it's the "refany" progr=
>am (now in a sibling directory to bddtest)
>>=20
>>   Mika
>>=20
>> "Rodney M. Bates" writes:
>>>=20
>>>=20
>>>> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>>>> Did you try using my bdd.tgz ?
>>>>=20
>>>> I am getting this error on:
>>>>=20
>>>> ancient PM3 on FreeBSD4
>>>>=20
>>>> head CM3 on AMD64_LINUX
>>>=20
>>> Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX s=
>ystem with
>>> a freshly updated head.  Hacking around, I tried to use the binaries in b=
>dd.tgz,
>>> but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files=
> are
>>> slightly different in size when I recompiled, but I think there is an exp=
>lanation
>>> for that.
>>>=20
>>> I can't build bddtest with the release compiler, because it needs caltech=
>_parser,
>>> and that won't build in the release.  This combination of [non]failures s=
>eems
>>> pretty strange.  I am also wondering about a problem with type checks in t=
>he
>>> runtime.
>>>=20
>>>=20
>>>>=20
>>>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>>>=20
>>>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.=
>
>>>>=20
>>>> I already tried rearranging the files (the implementation and interfaces=
>
>>>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>>>> right now).  With no effect.  I don't know what it is about my code that=
>'s
>>>> tripping up the compiler.
>>>>=20
>>>> It's also a "decent" BDD package so someone (me I guess, but anyone
>>>> else who's interested is welcome to) might want to add it to CM3... :-)
>>>> Kind of a pain if you can't print out the debugging info though!
>>>> (Actually the workarounds are OK.)
>>>>=20
>>>>     Mika
>>>>=20
>>>> "Rodney M. Bates" writes:
>>>>>=20
>>>>>=20
>>>>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>>>> Were you able to get it to fail on any of your computers?
>>>>>=20
>>>>> No, the only times I got narrow failures were due to errors
>>>>> in my trial cases.
>>>>>=20
>>>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>>>> LINUXLIBC6 system.
>>>>>=20
>>>>>>=20
>>>>>> Your Z fails for me---CM3 and PM3 both.
>>>>>>=20
>>>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>>>=20
>>>>>>=20
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    An explicit or implicit NARROW() operation failed.
>>>>>> ***    file "../src/Main.m3", line 30
>>>>>> ***
>>>>>=20
>>>>> Well that torpedos both my theories.
>>>>>=20
>>>>> But it is hard to imagine that bad code generated for an implicit narro=
>w
>>>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>>>> might be more believable if it only happened when narrowing an actual
>>>>> parameter before passing it.
>>>>>=20
>>>>>>=20
>>>>>> Abort
>>>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>>>> Main.m3~  Main.m3
>>>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>>>       1  MODULE Main;
>>>>>>       2  IMPORT BDD, IO;
>>>>>>       3
>>>>>>       4  PROCEDURE P() : REFANY =3D
>>>>>>       5    BEGIN
>>>>>>       6      VAR x : REFANY;
>>>>>>       7      BEGIN
>>>>>>       8        x :=3D BDD.New("a");
>>>>>>       9        Z(x);
>>>>>>      10        RETURN x
>>>>>>      11      END
>>>>>>      12    END P;
>>>>>>      13
>>>>>>      14  PROCEDURE Q() =3D
>>>>>>      15    BEGIN
>>>>>>      16      IO.Put(P() & "\n")
>>>>>>      17    END Q;
>>>>>>      18
>>>>>>      19  (*
>>>>>>      20  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      21    BEGIN
>>>>>>      22      TYPECASE x OF
>>>>>>      23        BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>      24      END;
>>>>>>      25    END Z;
>>>>>>      26  *)
>>>>>>      27  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      28     VAR b: BDD.T;
>>>>>>      29     BEGIN
>>>>>>      30       b :=3D x;
>>>>>>      31       x :=3D BDD.Format(b)
>>>>>>      32     END Z;
>>>>>>      33
>>>>>>      34
>>>>>>      35
>>>>>>      36  BEGIN
>>>>>>      37    Q()
>>>>>>      38  END Main.
>>>>>> (581)truffles:~/bsd/refany/src>
>>>>>> "Rodney M. Bates" writes:
>>>>>>> What does this do?
>>>>>>>=20
>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>    VAR b: BDD.T;
>>>>>>>    BEGIN
>>>>>>>      b :=3D x;
>>>>>>>      x :=3D BDD.Format(b)
>>>>>>>    END Z;
>>>>>>>=20
>>>>>>> The common property of the failing cases seems to be the necessity
>>>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>>>=20
>>>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>>>> with even more irrelevant (one would think) stuff pared out.  I moved=
>
>>>>>>> T, Root, and Format into main, removed all but one fields of T and Ro=
>ot,
>>>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body=
>
>>>>>>> from Format.
>>>>>>>=20
>>>>>>> Looking at your code, I agree it should work.
>>>>>>>=20
>>>>>>>=20
>>>>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>> a
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> This does NOT work:
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(x)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> ***
>>>>>>>> *** runtime error:
>>>>>>>> ***    NARROW failed
>>>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>>>> ***
>>>>>>>>=20
>>>>>>>>     use option @M3stackdump to get a stack trace
>>>>>>>> Abort
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Same result PM3 / CM3.
>>>>>>>>=20
>>>>>>>>        Mika
>>>>=20


From rodney_bates at lcwb.coop  Wed Jan 22 20:46:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Wed, 22 Jan 2014 13:46:55 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52E0202F.3020307@lcwb.coop>

Some more info:

The front end is generating incorrect narrow code, only
where the type T is an unrevealed <: REFANY.  When T and Root
are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
and implicit narrow in an assignment.

When opaque, for explicit and implicit narrow, it simply compares
the typecode of x to a single typecode (no doubt for T) and demands
they be equal.  In this case, the actual allocated type of x is Root,
a proper subtype of T.

For ISTYPE, it does call CheckIsType even when the types are opaque.

The fact that it happens only when the types are opaque lends
credibility to the fact that this bug has gone unnoticed at least
as far back as PM3.

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Wed Jan 22 21:47:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 22 Jan 2014 12:47:49 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E0202F.3020307@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
Message-ID: <20140122204749.C724E1A208F@async.async.caltech.edu>


Aha so declaring the type as <: ROOT will probably work as a workaround?

Thank you for looking into this.

"Rodney M. Bates" writes:
>Some more info:
>
>The front end is generating incorrect narrow code, only
>where the type T is an unrevealed <: REFANY.  When T and Root
>are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>and implicit narrow in an assignment.
>
>When opaque, for explicit and implicit narrow, it simply compares
>the typecode of x to a single typecode (no doubt for T) and demands
>they be equal.  In this case, the actual allocated type of x is Root,
>a proper subtype of T.
>
>For ISTYPE, it does call CheckIsType even when the types are opaque.
>
>The fact that it happens only when the types are opaque lends
>credibility to the fact that this bug has gone unnoticed at least
>as far back as PM3.


From jay.krell at cornell.edu  Thu Jan 23 08:51:58 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 07:51:58 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/046e93c5/attachment.html>

From rcolebur at SCIRES.COM  Thu Jan 23 12:59:34 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 11:59:34 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
Message-ID: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:

  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/ab6f0152/attachment.html>

From rodney_bates at lcwb.coop  Thu Jan 23 16:25:59 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 23 Jan 2014 09:25:59 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140122204749.C724E1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
Message-ID: <52E13487.9090504@lcwb.coop>



On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
> Aha so declaring the type as <: ROOT will probably work as a workaround?
>

I'm not sure whether that would do it or not.  In my pared-down case, I had
moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
the failures go away.

Until it is fixed, I think the best workaround is TYPECASE with a binding:

TYPECASE x
OF BDD.T(b) =>
   Format(b), etc.

This is known to be working now.  You've probably thought about this, but
this way requires only a single runtime type test.  Either TYPECASE without
the binding of b, or ISTYPE, will require an additional recheck of the same
thing when the narrow, implicit or explicit, happens.  I doubt even a
pretty good optimizing compiler could remove this.  For ISTYPE, it would have
to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
relation between CheckIsType and ScanTypecase.

> Thank you for looking into this.
>
> "Rodney M. Bates" writes:
>> Some more info:
>>
>> The front end is generating incorrect narrow code, only
>> where the type T is an unrevealed <: REFANY.  When T and Root
>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>> and implicit narrow in an assignment.
>>
>> When opaque, for explicit and implicit narrow, it simply compares
>> the typecode of x to a single typecode (no doubt for T) and demands
>> they be equal.  In this case, the actual allocated type of x is Root,
>> a proper subtype of T.
>>
>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>
>> The fact that it happens only when the types are opaque lends
>> credibility to the fact that this bug has gone unnoticed at least
>> as far back as PM3.
>



From hosking at cs.purdue.edu  Thu Jan 23 17:51:02 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 23 Jan 2014 11:51:02 -0500
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
Message-ID: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> Jay,
> 
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
> 
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
> 
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
> 
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
> 
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
> 
>>   There is a behavior/bug in wow64.  
>>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
>> 
>> 
>> 
>>   SuspendThread / GetThreadContext work like this:  
>> 
>> 
>>   32bit processes consist almost entirely of 32bit code.  
>>   There is a small amount of 64bit code.  
>> 
>> 
>>   If you suspend while running 32bit code, GetThreadContext works.  
>>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
>> 
>> 
>>   64bit code is run en route to syscalls.  
>>   For example you call:  
>>     1 kernel32!Sleep   
>>     2 it calls 32bit NtDelayExecution   
>>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>>     4 which calls native NtDelayExecuation   
>> 
>>  
>>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>>     You can step through it very easily in a debugger. Really. 
>>   Where GetThreadContext knows where to get it.  
>>   The problem is that saving context is not atomic.  
>>   You can suspend while saving context.  
>> 
>> 
>>   What to do? 
>> 
>> 
>>   scratch/wow64stack contains a program that detects the bug.
>>   I believe it is the basis of a workaround for the bug.   
>> 
>> 
>>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>>   not only functions that use exception handling, but also functions that call "extern" functions  
>>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>>   use that, if it isn't zero. 
>>   Normally it will be zero.  
>>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>>   That is why it isn't a set/set-zero pattern (like in the test program).   
>> 
>> 
>>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
>> 
>> 
>>   Rejected counter proposal: 
>>    Don't suspend/gc when running a syscall.  
>>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
>> 
>> 
>>  Possible augmentation: if running native, short circuit most of this 
>> 
>> 
>>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
>> 
>> 
>>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
>> 
>> 
>>   Performance impact: hypothetically large but probably not noticable. 
>> 
>> 
>>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>>   We could further augment pragmas to discern them. 
>> 
>> 
>>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
>> 
>> 
>> 
>>   Agreed? I'll make the compiler change? 
>> 
>> 
>> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
>> 
>> 
>> Eventually cooperative suspend will cause this to fall away as a problem..
>> 
>> 
>>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/8a7dd0a9/attachment.html>

From jay.krell at cornell.edu  Thu Jan 23 21:03:09 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 20:03:09 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>,
	<COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
Message-ID: <COL130-W24D43515B172BD96CF08F6E6A60@phx.gbl>

Syscalls are "doubly wrapped".
Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention.
Lesser known, there are also some syscalls in gdi32.dll and user32.dll.
None of them are public interfaces.
They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll.
For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change.
Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work.

There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall.

Bottom line is though, one isn't really supposed to know where the syscalls are.

 - Jay

From: jay.krell at cornell.edu
To: hosking at cs.purdue.edu; rcolebur at scires.com
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
Date: Thu, 23 Jan 2014 19:57:27 +0000




There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/7b51ac64/attachment.html>

From jay.krell at cornell.edu  Thu Jan 23 20:57:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 19:57:27 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
Message-ID: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5f6fec82/attachment.html>

From rcolebur at SCIRES.COM  Thu Jan 23 23:23:43 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

When you say native 32/64-bit "should be ok", you really mean ok with respect to the SYSWOW64 problem you've identified.

Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay
________________________________
From: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>; jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484






On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:
  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/c534e985/attachment.html>

From jay.krell at cornell.edu  Fri Jan 24 00:09:50 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 23:09:50 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>

Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?









Jay:
 
When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 

 
Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Thursday, January 23, 2014 2:57 PM

To: Tony; Coleburn, Randy

Cc: m3devel

Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?


 


There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.

Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.



 - Jay





From:
hosking at cs.purdue.edu

Date: Thu, 23 Jan 2014 11:51:02 -0500

To: rcolebur at SCIRES.COM

CC: m3devel at elegosoft.com; 
jay.krell at cornell.edu

Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?



Yes, agreed.

We need to address the native thread problem on all platforms (both pthread and windows).


 


However, this particular problem is pernicious if we need to save M3 stack state at all external calls.


 


Are there any linker tricks for Windows one can use to wrap only system calls?


 


I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,
  This approach is used for all modern Java implementations that I am aware of.


 


It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread
 state.


 




Antony Hosking | Associate
 Professor | Computer Science | Purdue University


305 N. University Street | West Lafayette | IN 47907 | USA


Mobile +1
 765 427 5484


 














 


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


Jay,


 


If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 


 


Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both
 32 & 64 bit platforms. 


 


Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 


 


Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?


 


--Randy



Sent from my iPhone





On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:



  There is a behavior/bug in wow64.  

  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.







  SuspendThread / GetThreadContext work like this:  





  32bit processes consist almost entirely of 32bit code.  

  There is a small amount of 64bit code.  





  If you suspend while running 32bit code, GetThreadContext works.  

  if you suspend while running 64bit code, GetThreadContext usually but not always works.  





  64bit code is run en route to syscalls.  

  For example you call:  

    1 kernel32!Sleep   

    2 it calls 32bit NtDelayExecution   

    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 

    4 which calls native NtDelayExecuation   



 

  In between 2 and 3, within 64bit code, the 32bit context is saved.  

    You can step through it very easily in a debugger. Really. 

  Where GetThreadContext knows where to get it.  

  The problem is that saving context is not atomic.  

  You can suspend while saving context.  





  What to do? 





  scratch/wow64stack contains a program that detects the bug.

  I believe it is the basis of a workaround for the bug.   





  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 

  not only functions that use exception handling, but also functions that call "extern" functions  

  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection

  use that, if it isn't zero. 

  Normally it will be zero.  

  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  

  That is why it isn't a set/set-zero pattern (like in the test program).   





  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 





  Rejected counter proposal: 

   Don't suspend/gc when running a syscall.  

   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  





 Possible augmentation: if running native, short circuit most of this 





  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 

   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 





   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 

   until/unless there is another 32bit platform runable on some similar 64bit platforms. 





  Performance impact: hypothetically large but probably not noticable. 





  Furthe refinements: It isn't extern/native code per se, it is syscalls.

  We could further augment pragmas to discern them. 





  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 

  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 

  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 







  Agreed? I'll make the compiler change? 





Oh, also, not just stack pointer, but other registers, at least non-volatiles?





Eventually cooperative suspend will cause this to fall away as a problem..





 - Jay





 



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5aed95c6/attachment.html>

From adacore at marino.st  Fri Jan 24 00:42:01 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 00:42:01 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBCDBA.8030809@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>
Message-ID: <52E1A8C9.3010903@marino.st>

On 1/19/2014 14:06, John Marino wrote:
> On 1/19/2014 13:50, Jay K wrote:
>> And there are advantages not yet implemented, in particular, efficient
>> exception handling by generating C++.
> 
> Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> dl_iterate_phdr which gcc uses for efficient zero-cost exception
> handling.  That might come in handy here (I implemented it in
> DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> 
> 
>> Your patches should apply fairly cleanly to any new checkout.
>> I am willing to through them and apply them to CVS if you want --
>> depends on if you are ok with my stealing your credit, vs. you want to
>> commit them yourselves.
> 
> I would love it if you did this - I'm happy with a mention in the commit
> message.  I just attached a tarball of all the patches I'm using right
> now.  Some of these changes are specifically for the cross compiler (I'm
> think of the /scripts changes mostly) but most are valid assuming they
> still apply to the head of the repo.  Please take as much as you want
> with my gratitude.
> 
>> I already skimmed them and they all look easy.
>> DragonflyBSD is basically FreeBSD by another name.
> 
> That's not an accident.  As a DragonFly developer that mostly works on
> userland, I've been converging FreeBSD and DragonFly again.  They
> drifted apart after 10 years, but I thought it better to keep the
> userlands as similar as possible for 3rd party applications.
> 
>> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
>> But the ABI is almost the same -- heck, for the most part,
>> Linux==NetBSD==FreeBSD==OpenBSD.
>> Sure, they might be implemented differently, but they are highly source
>> compatible and every highly object code compatible.
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining, and it was built by the latest patches.
>> Agreed, we are both missing something simple.
>> Usually this is the error you get when you don't upgrade
>> Target.i3/Target.m3.
>> If you want to press on with the current approach and ignore my advise,
> 
> I'm not ignoring it; it's more like I'm not quite ready to give up since
> I'm so close.  I think I could actually get these libs and cm3 to build
> on DragonFly if I used AMD64_FREEBSD as the target with the
> cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> I'd then get stuck on the DragonFly side trying to compile cm3 with
> itself, theoretically I should get the same error there.
> 
> 
>> Ugh. I saw something about that in your diffs. Definitely we intend to
>> link statically.
>> I actually went to the trouble of removing use of gmp/mpfr/mpc in
>> current source.
>> They are used for actually very little and aren't worth it.
> 

Hi Jay,
If you get those patches committed and publish an official snapshot
(compressed tarball) with those changes in it, then I'll update the
FreeBSD port to that snapshot.

If new patches are required, say to convert FreeBSD to use the C
backend, I'll feed those back too.

Then I'll try porting DragonFly again, your way.
Thanks,
John





From jay.krell at cornell.edu  Fri Jan 24 09:45:39 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 08:45:39 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E1A8C9.3010903@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
Message-ID: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>

sorry, I'm delayed..so much to do..
I commited the networking fix..
The m3core/unix one I think we already have.

Later...
 - Jay

> Date: Fri, 24 Jan 2014 00:42:01 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 14:06, John Marino wrote:
> > On 1/19/2014 13:50, Jay K wrote:
> >> And there are advantages not yet implemented, in particular, efficient
> >> exception handling by generating C++.
> > 
> > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> > dl_iterate_phdr which gcc uses for efficient zero-cost exception
> > handling.  That might come in handy here (I implemented it in
> > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> > 
> > 
> >> Your patches should apply fairly cleanly to any new checkout.
> >> I am willing to through them and apply them to CVS if you want --
> >> depends on if you are ok with my stealing your credit, vs. you want to
> >> commit them yourselves.
> > 
> > I would love it if you did this - I'm happy with a mention in the commit
> > message.  I just attached a tarball of all the patches I'm using right
> > now.  Some of these changes are specifically for the cross compiler (I'm
> > think of the /scripts changes mostly) but most are valid assuming they
> > still apply to the head of the repo.  Please take as much as you want
> > with my gratitude.
> > 
> >> I already skimmed them and they all look easy.
> >> DragonflyBSD is basically FreeBSD by another name.
> > 
> > That's not an accident.  As a DragonFly developer that mostly works on
> > userland, I've been converging FreeBSD and DragonFly again.  They
> > drifted apart after 10 years, but I thought it better to keep the
> > userlands as similar as possible for 3rd party applications.
> > 
> >> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> >> But the ABI is almost the same -- heck, for the most part,
> >> Linux==NetBSD==FreeBSD==OpenBSD.
> >> Sure, they might be implemented differently, but they are highly source
> >> compatible and every highly object code compatible.
> >>> I must be misunderstanding what you mean by "upgrade". It's the
> >>> cross-compiler that's complaining, and it was built by the latest patches.
> >> Agreed, we are both missing something simple.
> >> Usually this is the error you get when you don't upgrade
> >> Target.i3/Target.m3.
> >> If you want to press on with the current approach and ignore my advise,
> > 
> > I'm not ignoring it; it's more like I'm not quite ready to give up since
> > I'm so close.  I think I could actually get these libs and cm3 to build
> > on DragonFly if I used AMD64_FREEBSD as the target with the
> > cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> > I'd then get stuck on the DragonFly side trying to compile cm3 with
> > itself, theoretically I should get the same error there.
> > 
> > 
> >> Ugh. I saw something about that in your diffs. Definitely we intend to
> >> link statically.
> >> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> >> current source.
> >> They are used for actually very little and aren't worth it.
> > 
> 
> Hi Jay,
> If you get those patches committed and publish an official snapshot
> (compressed tarball) with those changes in it, then I'll update the
> FreeBSD port to that snapshot.
> 
> If new patches are required, say to convert FreeBSD to use the C
> backend, I'll feed those back too.
> 
> Then I'll try porting DragonFly again, your way.
> Thanks,
> John
> 
> 
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9f8f3f93/attachment.html>

From adacore at marino.st  Fri Jan 24 10:11:02 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 10:11:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>, <52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
Message-ID: <52E22E26.10402@marino.st>

On 1/24/2014 09:45, Jay K wrote:
> sorry, I'm delayed..so much to do..
> I commited the networking fix..
> The m3core/unix one I think we already have.
> 
> Later...
>  - Jay


Right, but there's no downloadable source snapshot.  And if the
DragonFly support isn't in there, I'll have to carry all those patches
too.  So my goal is to have a few patches as possible.

Anyway, I have to base the port on something -- pulling latest SVN is
rarely done, and CVS is never done.  Some github projects don't make
releases and needed to be pulled directly, but this requires first git
to be built (lots of dependencies) and then github tweaks their service
and everything fails.  So I'm a big believer in static compressed
tarballs.

Which means I can't update FreeBSD cm3 version until one exists.
theoretically I can package one myself, but it looks better if it's
"official" and comes from one of the known m3 websites.

Thanks,
John


From jay.krell at cornell.edu  Fri Jan 24 10:59:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 09:59:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E22E26.10402@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>,
	<52E22E26.10402@marino.st>
Message-ID: <COL130-W58042B8075CEF3E2C8165CE6A10@phx.gbl>

Please be patient. We'll get a snapshot, soon. A release, unclear.

  - Jay

> Date: Fri, 24 Jan 2014 10:11:02 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/24/2014 09:45, Jay K wrote:
> > sorry, I'm delayed..so much to do..
> > I commited the networking fix..
> > The m3core/unix one I think we already have.
> > 
> > Later...
> >  - Jay
> 
> 
> Right, but there's no downloadable source snapshot.  And if the
> DragonFly support isn't in there, I'll have to carry all those patches
> too.  So my goal is to have a few patches as possible.
> 
> Anyway, I have to base the port on something -- pulling latest SVN is
> rarely done, and CVS is never done.  Some github projects don't make
> releases and needed to be pulled directly, but this requires first git
> to be built (lots of dependencies) and then github tweaks their service
> and everything fails.  So I'm a big believer in static compressed
> tarballs.
> 
> Which means I can't update FreeBSD cm3 version until one exists.
> theoretically I can package one myself, but it looks better if it's
> "official" and comes from one of the known m3 websites.
> 
> Thanks,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9ee26689/attachment.html>

From jay.krell at cornell.edu  Fri Jan 24 11:38:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 10:38:01 +0000
Subject: [M3devel] garbage collector compacting and volatile?
Message-ID: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>

Is our garbage collector compacting?


In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?
Update references anywhere? Just the stack? Also context?


There are tradeoffs either way.


If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?


If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.


If we read and write registers in the collector, then nothing has to be volatile.


Should this all be parameters passed to the backend?
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.


If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.


Thank you,
 - Jay

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/e86b30bc/attachment.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:24:35 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:24:35 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
Message-ID: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.
Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:

> Yes. Agreed.
> They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
> I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
>  
>  
>  - Jay
> 
>  
> From: rcolebur at SCIRES.COM
> To: m3devel at elegosoft.com
> Date: Thu, 23 Jan 2014 22:23:43 +0000
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> Jay:
>  
> When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 
>  
> Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Thursday, January 23, 2014 2:57 PM
> To: Tony; Coleburn, Randy
> Cc: m3devel
> Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
>  
> There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
> Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.
> 
>  - Jay
> From: hosking at cs.purdue.edu
> Date: Thu, 23 Jan 2014 11:51:02 -0500
> To: rcolebur at SCIRES.COM
> CC: m3devel at elegosoft.com; jay.krell at cornell.edu
> Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
> 
> Yes, agreed.
> We need to address the native thread problem on all platforms (both pthread and windows).
>  
> However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
>  
> Are there any linker tricks for Windows one can use to wrap only system calls?
>  
> I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
>  
> It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.
>  
> Antony Hosking | Associate Professor | Computer Science | Purdue University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Mobile +1 765 427 5484
>  
> 
> 
> 
> 
>  
> On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:
>  
> Jay,
>  
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
>  
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
>  
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
>  
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
>  
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
>   There is a behavior/bug in wow64.  
>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
> 
> 
> 
>   SuspendThread / GetThreadContext work like this:  
> 
> 
>   32bit processes consist almost entirely of 32bit code.  
>   There is a small amount of 64bit code.  
> 
> 
>   If you suspend while running 32bit code, GetThreadContext works.  
>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
> 
> 
>   64bit code is run en route to syscalls.  
>   For example you call:  
>     1 kernel32!Sleep   
>     2 it calls 32bit NtDelayExecution   
>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>     4 which calls native NtDelayExecuation   
> 
>  
>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>     You can step through it very easily in a debugger. Really. 
>   Where GetThreadContext knows where to get it.  
>   The problem is that saving context is not atomic.  
>   You can suspend while saving context.  
> 
> 
>   What to do? 
> 
> 
>   scratch/wow64stack contains a program that detects the bug.
>   I believe it is the basis of a workaround for the bug.   
> 
> 
>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>   not only functions that use exception handling, but also functions that call "extern" functions  
>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>   use that, if it isn't zero. 
>   Normally it will be zero.  
>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>   That is why it isn't a set/set-zero pattern (like in the test program).   
> 
> 
>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
> 
> 
>   Rejected counter proposal: 
>    Don't suspend/gc when running a syscall.  
>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
> 
> 
>  Possible augmentation: if running native, short circuit most of this 
> 
> 
>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
> 
> 
>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
> 
> 
>   Performance impact: hypothetically large but probably not noticable. 
> 
> 
>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>   We could further augment pragmas to discern them. 
> 
> 
>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
> 
> 
> 
>   Agreed? I'll make the compiler change? 
> 
> 
> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
> 
> 
> Eventually cooperative suspend will cause this to fall away as a problem..
> 
> 
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/df5802c0/attachment.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:12:33 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:12:33 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
Message-ID: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:

> Is our garbage collector compacting?

Yes, it is mostly-copying.
Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).

> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?

It reads both registers and stack.

> Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.
Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.

> There are tradeoffs either way.

Indeed. Too many to enumerate here.

> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).

> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.

No updates to values on the stacks.

> If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.

> Should this all be parameters passed to the backend?

Nope.

> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.

Nope.

> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.


It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/12b01775/attachment.html>

From jay.krell at cornell.edu  Sat Jan 25 05:04:59 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 25 Jan 2014 04:04:59 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
Message-ID: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>

I might yet have another solution here..there might be a way to detect the race and retry...later..

 - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:24:35 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified.  Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - JayFrom: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 



 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.  Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.  Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.  Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140125/b19a595a/attachment.html>

From hendrik at topoi.pooq.com  Sat Jan 25 05:38:03 2014
From: hendrik at topoi.pooq.com (Hendrik Boom)
Date: Fri, 24 Jan 2014 23:38:03 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
Message-ID: <20140125043803.GA3423@topoi.pooq.com>

On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> I might yet have another solution here..there might be a way to 
> detect the race and retry...later..
> 
>  - Jay

I seem to remember a lot of comments in the trestle code tht seem to 
be intended as assertions to catch race conditions.
Is any of that stuff conceivably helpful here?

-- hendrik


From jay.krell at cornell.edu  Sun Jan 26 07:08:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <20140125043803.GA3423@topoi.pooq.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>,
	<20140125043803.GA3423@topoi.pooq.com>
Message-ID: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>

No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140126/e01d9d37/attachment.html>

From jay.krell at cornell.edu  Mon Jan 27 08:23:26 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:23:26 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	, <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>, ,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>, ,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>, ,
	<20140125043803.GA3423@topoi.pooq.com>,
	<COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
Message-ID: <COL130-W141C96275EF6018B1D2D0E6A20@phx.gbl>


 > I might yet have another solution here
And it might only be for Windows 8 or newer.I tend to think we should go with the earlier proposal -- save context ourselves in thread locals when calling externals. Likely portable implementation is setjmp or getcontext, though we can maybe do better.This would be part of cooperative suspend anyway, which we want to move to.

 - Jay

From: jay.krell at cornell.edu
To: hendrik at topoi.pooq.com; m3devel at elegosoft.com
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?




No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/146c0737/attachment.html>

From jay.krell at cornell.edu  Mon Jan 27 08:45:08 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:45:08 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
Message-ID: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>

In summary, shortest:

  We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).    We must be able to read the stack.    We need not write registers or stack (from garbage collector).  

Slight elaboration: Copying/compaction only occurs when references are only from globals and heap.  Therefore updates/writes only occur to globals and heap. 

 Is there any desire to do better/different?  To require the ability to update stack/registers?  What does Java typically to? 


I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.To do better/different would require more backend cooperation.We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Thanks, - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:12:33 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] garbage collector compacting and volatile?

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
Is our garbage collector compacting?

Yes, it is mostly-copying.Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?

It reads both registers and stack.
Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
There are tradeoffs either way.

Indeed. Too many to enumerate here.
If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.

No updates to values on the stacks.
If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.
Should this all be parameters passed to the backend?

Nope.
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.

Nope.
If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.

It is mostly-copying.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/9dd883b4/attachment.html>

From hosking at cs.purdue.edu  Mon Jan 27 16:16:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 27 Jan 2014 10:16:57 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
Message-ID: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>



On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:

> In summary, shortest:
> 
> 
>   We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).  
>   We must be able to read the stack.  
>   We need not write registers or stack (from garbage collector).  

Correct.

> Slight elaboration:
>  Copying/compaction only occurs when references are only from globals and heap. 
>  Therefore updates/writes only occur to globals and heap. 

Correct.

>  Is there any desire to do better/different? 
>  To require the ability to update stack/registers? 

Without control of the register allocator, optimisations, and stack layout performed by the compiler (especially back-end) getting fully accurate stack maps is very difficult.

>  What does Java typically to? 

Java VMs usually control their own destiny, having full control of JIT and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.

> I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.

Correct.

> Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.

Correct.

> To do better/different would require more backend cooperation.
> We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Correct.

> 
> 
> Thanks,
>  - Jay
> 
> From: hosking at cs.purdue.edu
> Date: Fri, 24 Jan 2014 13:12:33 -0500
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
> 
> Is our garbage collector compacting?
> 
> Yes, it is mostly-copying.
> Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
> 
> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?
> 
> It reads both registers and stack.
> 
> Update references anywhere? Just the stack? Also context?
> 
> The collector first pins all pages referenced from the stacks, including registers.
> Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
> 
> There are tradeoffs either way.
> 
> Indeed. Too many to enumerate here.
> 
> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?
> 
> The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
> 
> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.
> 
> No updates to values on the stacks.
> 
> If we read and write registers in the collector, then nothing has to be volatile.
> 
> Read only.  No writes to registers.
> 
> Should this all be parameters passed to the backend?
> 
> Nope.
> 
> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.
> 
> Nope.
> 
> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.
> 
> It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/6f9b0cfd/attachment.html>

From rodney_bates at lcwb.coop  Mon Jan 27 20:46:23 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 13:46:23 -0600
Subject: [M3devel] New compile failure
Message-ID: <52E6B78F.7000003@lcwb.coop>

The compiler in the head, with some recent changes, gives:

rodney at yellowstone:~/proj/m3/cm3-head/cm3/m3-sys/m3middle$ cm3

Fatal Error: configuration file didn't specify BUILD_DIR

This message is coming from cm3/src/Main.m3, which has recently
changed, but there are related changes to other source files
too.  Reverting Main.m3 to the previous version won't compile.

Putting an older executable for cm3 into the bin directory,
with recently updated and installed cm3.cfg and config works.


From rodney_bates at lcwb.coop  Mon Jan 27 21:49:43 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 14:49:43 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E13487.9090504@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
	<52E13487.9090504@lcwb.coop>
Message-ID: <52E6C667.6030708@lcwb.coop>

It should be fixed in the head, now.

On 01/23/2014 09:25 AM, Rodney M. Bates wrote:
>
>
> On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
>> Aha so declaring the type as <: ROOT will probably work as a workaround?
>>
>
> I'm not sure whether that would do it or not.  In my pared-down case, I had
> moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
> the failures go away.
>
> Until it is fixed, I think the best workaround is TYPECASE with a binding:
>
> TYPECASE x
> OF BDD.T(b) =>
>    Format(b), etc.
>
> This is known to be working now.  You've probably thought about this, but
> this way requires only a single runtime type test.  Either TYPECASE without
> the binding of b, or ISTYPE, will require an additional recheck of the same
> thing when the narrow, implicit or explicit, happens.  I doubt even a
> pretty good optimizing compiler could remove this.  For ISTYPE, it would have
> to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
> unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
> relation between CheckIsType and ScanTypecase.
>
>> Thank you for looking into this.
>>
>> "Rodney M. Bates" writes:
>>> Some more info:
>>>
>>> The front end is generating incorrect narrow code, only
>>> where the type T is an unrevealed <: REFANY.  When T and Root
>>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>>> and implicit narrow in an assignment.
>>>
>>> When opaque, for explicit and implicit narrow, it simply compares
>>> the typecode of x to a single typecode (no doubt for T) and demands
>>> they be equal.  In this case, the actual allocated type of x is Root,
>>> a proper subtype of T.
>>>
>>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>>
>>> The fact that it happens only when the types are opaque lends
>>> credibility to the fact that this bug has gone unnoticed at least
>>> as far back as PM3.
>>
>
>



From jay.krell at cornell.edu  Tue Jan 28 00:13:46 2014
From: jay.krell at cornell.edu (Jay)
Date: Mon, 27 Jan 2014 15:13:46 -0800
Subject: [M3devel] Fwd: Why does std::chrono now() uses slow syscall?
References: <CAH6eHdSjowHv+AM=T7nKgyoVuthbg6dAVbBvP0+UBxg0w=_jgg@mail.gmail.com>
Message-ID: <9FD55748-9752-4E40-9605-78F98AE62A5D@gmail.com>

Hints for how m3core/libm3 should get time efficiently...

 - Jay

Begin forwarded message:

> From: Jonathan Wakely <jwakely.gcc at gmail.com>
> Date: January 26, 2014, 2:33:36 AM PST
> To: Keith Erickson <kerickso at pppl.gov>
> Cc: gcc-help <gcc-help at gcc.gnu.org>
> Subject: Re: Why does std::chrono now() uses slow syscall?
> 
> On 26 January 2014 07:50, Keith Erickson wrote:
>> My questions, then, are....  why would I ever want to use the syscall
>> version?  It seems that configure prefers it, and will use it if it's
>> available.
> 
> It prefers to use clock_gettime and only uses the syscall if a
> configure test using clock_gettime(CLOCK_MONOTONIC, &tp) fails.
> 
>> But it's stupidly slow, even on the fasted server CPU that
>> AMD sells (Opteron 6386 SE at the time of this writing).  How do I not
>> use this slow method?  Do I have to compile gcc specially?  What are
>> the drawbacks to forcing configure to not set that #define?  Is there
>> an approved way to tell configure to use a fast time?
> 
> Prior to glibc 2.17 clock_gettime was in librt.so not libc.so, which
> is not used automatically by libstdc++. To tell configure to use it
> you need to build GCC with --enable-libstdcxx-time=rt, but that means
> that libstdc++.so will be linked to librt.so which depends on
> libpthread.so which means that the library always thinks it is part of
> a multithreaded application and so even single-threaded programs use
> atomic operations for internal reference-counting (e.g. in std::string
> and std::shared_ptr).  That's why we don't use clock_gettime unless
> explicitly configured to do so.
> 
> With glibc 2.17 or later clock_gettime will be used automatically,
> because the functions were moved out of librt.so.
> 
> On my Fedora 20 system I get
> 
> | #define HAVE_SYS_TIME_H 1
> | #define _GLIBCXX_USE_GETTIMEOFDAY 1
> | #define _GLIBCXX_USE_CLOCK_MONOTONIC 1
> | #define _GLIBCXX_USE_CLOCK_REALTIME 1
> | #define _GLIBCXX_USE_SCHED_YIELD 1
> | #define _GLIBCXX_USE_NANOSLEEP 1
> 
> which means it doesn't need the syscall version.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/d9369ca7/attachment.html>

From mika at async.caltech.edu  Wed Jan 29 00:49:19 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 28 Jan 2014 15:49:19 -0800
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
Message-ID: <20140128234919.0E0C81A208F@async.async.caltech.edu>


Sounds like Jay doesn't need to buy a copy of your book :-)

Tony Hosking writes:
>
>--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=iso-8859-1
>
>
>
>On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>
>> In summary, shortest:
>>=20
>>=20
>>   We must be able to read registers (or save them to the stack or =
>elsewhere (known thread locals)). =20
>>   We must be able to read the stack. =20
>>   We need not write registers or stack (from garbage collector). =20
>
>Correct.
>
>> Slight elaboration:
>>  Copying/compaction only occurs when references are only from globals =
>and heap.=20
>>  Therefore updates/writes only occur to globals and heap.=20
>
>Correct.
>
>>  Is there any desire to do better/different?=20
>>  To require the ability to update stack/registers?=20
>
>Without control of the register allocator, optimisations, and stack =
>layout performed by the compiler (especially back-end) getting fully =
>accurate stack maps is very difficult.
>
>>  What does Java typically to?=20
>
>Java VMs usually control their own destiny, having full control of JIT =
>and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>
>> I guess there is the problem that we don't know for certain if values =
>in registers and stack are pointers, or just coincident integers, and so =
>updating them can be wrong.
>
>Correct.
>
>> Whereas in the globals and "reachable thereof" we have precise type =
>information for, so we can update those values safely.
>
>Correct.
>
>> To do better/different would require more backend cooperation.
>> We conservatively treat registers/stack as pointers if they look like =
>them, and therefore keep values alive, but not trust them so much as to =
>update them.
>
>Correct.


From jay.krell at cornell.edu  Wed Jan 29 01:00:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 29 Jan 2014 00:00:20 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>, ,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>,
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>,
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>,
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <COL130-W51823A67DAA7B4D7B7BDD3E6AC0@phx.gbl>

Good point.  I have it around somewhere... (I need that automatic resource management in real life..)
 
 - Jay

 
> To: hosking at cs.purdue.edu
> Date: Tue, 28 Jan 2014 15:49:19 -0800
> From: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
> >
> >--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
> >Content-Transfer-Encoding: quoted-printable
> >Content-Type: text/plain;
> >	charset=iso-8859-1
> >
> >
> >
> >On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
> >
> >> In summary, shortest:
> >>=20
> >>=20
> >>   We must be able to read registers (or save them to the stack or =
> >elsewhere (known thread locals)). =20
> >>   We must be able to read the stack. =20
> >>   We need not write registers or stack (from garbage collector). =20
> >
> >Correct.
> >
> >> Slight elaboration:
> >>  Copying/compaction only occurs when references are only from globals =
> >and heap.=20
> >>  Therefore updates/writes only occur to globals and heap.=20
> >
> >Correct.
> >
> >>  Is there any desire to do better/different?=20
> >>  To require the ability to update stack/registers?=20
> >
> >Without control of the register allocator, optimisations, and stack =
> >layout performed by the compiler (especially back-end) getting fully =
> >accurate stack maps is very difficult.
> >
> >>  What does Java typically to?=20
> >
> >Java VMs usually control their own destiny, having full control of JIT =
> >and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
> >
> >> I guess there is the problem that we don't know for certain if values =
> >in registers and stack are pointers, or just coincident integers, and so =
> >updating them can be wrong.
> >
> >Correct.
> >
> >> Whereas in the globals and "reachable thereof" we have precise type =
> >information for, so we can update those values safely.
> >
> >Correct.
> >
> >> To do better/different would require more backend cooperation.
> >> We conservatively treat registers/stack as pointers if they look like =
> >them, and therefore keep values alive, but not trust them so much as to =
> >update them.
> >
> >Correct.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140129/895ab0a2/attachment.html>

From hosking at cs.purdue.edu  Wed Jan 29 01:07:15 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 28 Jan 2014 19:07:15 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <38455303-D2F4-4BDD-8E31-D7139D566436@cs.purdue.edu>

;-)

I should send him a copy!

On Jan 28, 2014, at 6:49 PM, mika at async.caltech.edu wrote:

> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
>> 
>> --Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=iso-8859-1
>> 
>> 
>> 
>> On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>> 
>>> In summary, shortest:
>>> =20
>>> =20
>>>  We must be able to read registers (or save them to the stack or =
>> elsewhere (known thread locals)). =20
>>>  We must be able to read the stack. =20
>>>  We need not write registers or stack (from garbage collector). =20
>> 
>> Correct.
>> 
>>> Slight elaboration:
>>> Copying/compaction only occurs when references are only from globals =
>> and heap.=20
>>> Therefore updates/writes only occur to globals and heap.=20
>> 
>> Correct.
>> 
>>> Is there any desire to do better/different?=20
>>> To require the ability to update stack/registers?=20
>> 
>> Without control of the register allocator, optimisations, and stack =
>> layout performed by the compiler (especially back-end) getting fully =
>> accurate stack maps is very difficult.
>> 
>>> What does Java typically to?=20
>> 
>> Java VMs usually control their own destiny, having full control of JIT =
>> and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>> 
>>> I guess there is the problem that we don't know for certain if values =
>> in registers and stack are pointers, or just coincident integers, and so =
>> updating them can be wrong.
>> 
>> Correct.
>> 
>>> Whereas in the globals and "reachable thereof" we have precise type =
>> information for, so we can update those values safely.
>> 
>> Correct.
>> 
>>> To do better/different would require more backend cooperation.
>>> We conservatively treat registers/stack as pointers if they look like =
>> them, and therefore keep values alive, but not trust them so much as to =
>> update them.
>> 
>> Correct.



From mika at async.caltech.edu  Wed Jan  1 11:48:36 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 01 Jan 2014 02:48:36 -0800
Subject: [M3devel] mklib not compiling
Message-ID: <20140101104836.E13C51A208E@async.async.caltech.edu>

Hi m3devel,

Trying to upgrade my compiler on AMD64_LINUX...

=== package /home/mika/cm3-cvs-anon/cm3/m3-sys/mklib ===
 +++ cm3 -build -DROOT='/home/mika/cm3-cvs-anon/cm3' $RARGS  && cm3 -ship $RARGS -DROOT='/home/mika/cm3-cvs-anon/cm3'  +++
--- building in AMD64_LINUX ---

ignoring ../src/m3overrides

new source -> compiling Main.m3
"../src/Main.m3", line 659: incompatible types (b)
"../src/Main.m3", line 660: incompatible types (b)
"../src/Main.m3", line 661: incompatible types (b)
"../src/Main.m3", line 662: incompatible types (b)
"../src/Main.m3", line 663: incompatible types (b)
"../src/Main.m3", line 664: incompatible types (b)
"../src/Main.m3", line 665: incompatible types (b)
7 errors encountered
compilation failed => not building program "mklib"
Fatal Error: package build failed

I'm at the head...

    Mika


From mika at async.caltech.edu  Thu Jan  2 17:34:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 08:34:10 -0800
Subject: [M3devel] building the CVS head---how???
Message-ID: <20140102163410.757DB1A2080@async.async.caltech.edu>


So here is the situation.

New machine, new problems.  

AMD64_LINUX.  I want to try the latest compiler, because there are issues with what I have.

What I have is the following:

Critical Mass Modula-3 version 5.8.6
  last updated: 2010-04-11
  compiled: 2010-07-12 20:10:34
  configuration: /home/mika/cm3.amd64-linux/bin/cm3.cfg
  host: AMD64_LINUX
  target: AMD64_LINUX

I think the boot archives on the CM3 web site are about the same age.

I try 

./boot1.py AMD64_LINUX

and I get

"../src/win32/WinSock.i3", line 662: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 665: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 688: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 691: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 694: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 697: unsupported language or calling convention (PASCAL)
34 errors encountered
new exporters -> recompiling WinVer.i3
"../src/win32/WinVer.i3", line 158: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 168: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 180: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 190: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 203: unsupported language or calling convention (APIENTRY)

(etc)

all right, it's broken, something needs to be fixed.

WHAT IS NOT ALL RIGHT:

At this point my regular cm3.cfg is messed up.  If I had been silly enough to try to bootstrap
a new compiler (for a different machine) without first saving my old CM3 installation, I'd
be screwed.

But from reading the mailing list logs, I am not sure if it is even possible to bootstrap
the head from the older installation.  How am I supposed to get a new compiler working?
Build it on my Raspberry Pi (which has a working, recent CM3 on it)?  (After backing up
my old installation, of course.)

And what's this?  Bad python?

(119)truffles:~/cm3-cvs-anon/cm3/scripts/python>./boot1.py AMD64_LINUX
Traceback (most recent call last):
  File "./boot1.py", line 5, in <module>
    import pylib
  File "/big/home2/mika/2/cm3-cvs/cm3/scripts/python/pylib.py", line 587, in <module>
    for a in os.popen(CM3 + " -version 2>" + DevNull):
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
(120)truffles:~/cm3-cvs-anon/cm3/scripts/python>python --version
Python 2.7.3

     Mika
     Mika


From mika at async.caltech.edu  Fri Jan  3 00:15:37 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 15:15:37 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
Message-ID: <20140102231537.B953A1A2080@async.async.caltech.edu>


So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing

./boot1.py c AMD64_LINUX

I get:

new source -> compiling PklActionSeq.i3
new source -> compiling ConvertPacking.m3
"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)


***
*** runtime error:
***    <*ASSERT*> failed.
***    file "../src/M3C.m3", line 5890
***

Aborted
 *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***


   5887 PROCEDURE shift_right(self: T; type: IType) =
   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
   5889 BEGIN
   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
   5891     shift_left_or_right(self, type, "shift_right", ">>");
   5892 END shift_right;

...

Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...

     Mika


From mika at async.caltech.edu  Fri Jan  3 06:12:52 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:12:52 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
In-Reply-To: <20140102174632.3169C41A@resin13.mta.everyone.net>
References: <20140102174632.3169C41A@resin13.mta.everyone.net>
Message-ID: <20140103051252.88FC61A2080@async.async.caltech.edu>

Well I managed to get it to work using the following roundabout procedure:

Since the only archives I could find on elegosoft for AMD64_LINUX were ancient,
I installed VirtualBox and FreeBSD/amd64 inside that.  FreeBSD/amd64 for some
reason has very recent archives (of the old "cminstall" kind).

Downloaded the FreeBSD archive, cminstalled it, and ran upgrade.py on the latest
sources.

Had to remove some code from ConvertPacking.m3 because Jay's C-backend can't compile
that for some reason.

ran boot1.py, which gave me a boot archive.  Note---if you've done this with boot1.py "c"
before, you have to delete all the derived dirs and start over, or you get a mess.

Moved the boot archive to the Linux machine.

Resurrected ConvertPacking.m3

Ran boot2.sh (or equiv) until I got everything built.

That worked.

So... some observations: 

.. I have a very recent build on ARM_LINUX, but wasn't able to cross-compile.  Word size
problem?

.. I now have three very recent builds: ARM_LINUX, AMD64_FREEBSD, and AMD64_LINUX.  What
precisely do I have to do to make boot archives out of these so other people might be spared
this painful process?  Very few archives on the elego site are up to date.  Some NT stuff and
some FreeBSD stuff.  Nothing else that I can see.

    Mika

"Rodney Bates" writes:
>
>
>-Rodney Bates
>
>--- mika at async.caltech.edu wrote:
>
>From: mika at async.caltech.edu
>To: m3devel at elegosoft.com
>Subject: [M3devel] booting for AMD64_LINUX, continued...
>Date: Thu, 02 Jan 2014 15:15:37 -0800
>
>
>So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing
>
>./boot1.py c AMD64_LINUX
>
>I get:
>
>new source -> compiling PklActionSeq.i3
>new source -> compiling ConvertPacking.m3
>"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
>"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)
>
>
>***
>*** runtime error:
>***    <*ASSERT*> failed.
>***    file "../src/M3C.m3", line 5890
>***
>
>Aborted
> *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***
>
>
>   5887 PROCEDURE shift_right(self: T; type: IType) =
>   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
>   5889 BEGIN
>   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
>   5891     shift_left_or_right(self, type, "shift_right", ">>");
>   5892 END shift_right;
>
>...
>
>Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...
>
>I have one, but am out of the country right now and have no access to it.  
>I am waiting as I write to board the first airplane home, so will try to
>tar one up for you when I get there. 
>
>
>     Mika
>


From mika at async.caltech.edu  Fri Jan  3 06:57:18 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:57:18 -0800
Subject: [M3devel] pthreads still broken
Message-ID: <20140103055718.B12821A2080@async.async.caltech.edu>

On my new AMD64_LINUX build...

The thread tester still crashes.

First, it prints a couple of extraneous diagnostics.  These look like
they come from the threading code?

Then it segfaults.

(141)truffles:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX>gdb threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest...done.
(gdb) run 
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x2aaaab6f8700 (LWP 23996)]
[New Thread 0x2aaaab8f9700 (LWP 23997)]
[New Thread 0x2aaaabafa700 (LWP 23998)]
done
Creating fork threads...[New Thread 0x2aaaabcfb700 (LWP 23999)]
[New Thread 0x2aaaabefc700 (LWP 24000)]
[New Thread 0x2aaaac0fd700 (LWP 24001)]
done
Creating alloc threads...[New Thread 0x2aaaac2fe700 (LWP 24002)]
[New Thread 0x2aaaac4ff700 (LWP 24003)]
[New Thread 0x2aaaac700700 (LWP 24004)]
done
Creating lock threads...[New Thread 0x2aaaac901700 (LWP 24005)]
[New Thread 0x2aaaacb02700 (LWP 24006)]
[New Thread 0x2aaaacd03700 (LWP 24007)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x2aaaac901700 (LWP 24005)]
0x00002aaaaaf5b64b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x2aaaab6f8700 (LWP 23996)]
0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
44          IF (rd.buff = NIL) THEN
(gdb) where
#0  0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
#1  0x0000000000416d3a in FileRd__Open (M3_Bd56fi_p=<error reading variable>) at ../src/rw/FileRd.m3:16
#2  0x0000000000405424 in Main__RApply (M3_AP7a1g_cl=<error reading variable>) at ../src/Main.m3:182
#3  0x000000000044c23b in ThreadPThread__RunThread (M3_DMxDjQ_me=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:449
#4  0x000000000044bef8 in ThreadPThread__ThreadBase (M3_AJWxb1_param=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#5  0x00002aaaaaf56b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00002aaaab246a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x0000000000000000 in ?? ()
(gdb) 

The reason I even ran it was because I found that running the parallel compiler backend was unstable. 
  
Do user threads work on AMD64_LINUX?

     Mika


From mika at async.caltech.edu  Fri Jan  3 17:13:43 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 08:13:43 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
Message-ID: <20140103161343.F25291A2080@async.async.caltech.edu>

(78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LINUX/threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest...done.
(gdb) handle SIGILL 
Signal        Stop      Print   Pass to program Description
SIGILL        Yes       Yes     Yes             Illegal instruction
(gdb) handle SIGILL  nostop noprint
Signal        Stop      Print   Pass to program Description
SIGILL        No        No      Yes             Illegal instruction
(gdb) run
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
[New Thread 0x40bf4450 (LWP 2942)]
[New Thread 0x40df4450 (LWP 2943)]
done
Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
[New Thread 0x411f4450 (LWP 2945)]
[New Thread 0x413f4450 (LWP 2946)]
done
Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
[New Thread 0x417f4450 (LWP 2948)]
[New Thread 0x419f4450 (LWP 2949)]
done
Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
[New Thread 0x41df4450 (LWP 2951)]
[New Thread 0x41ff4450 (LWP 2952)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x41ff4450 (LWP 2952)]
0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "../src/runtime/common/RTCollector.m3", line 418
***


Program received signal SIGABRT, Aborted.
[Switching to Thread 0x40ff4450 (LWP 2944)]
0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) where
#0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
#2  0x00000000 in ?? ()
(gdb) 

It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
ARM_LINUX but would be surprised if it didn't work.

Yes the pthreads seem "more reliable" than they used to be, but I would
never use them in any sort of mission-critical application.  I use
Modula-3 programs for (among other things) financial transactions and
real-time motor controllers...

    Mika


From mika at async.caltech.edu  Fri Jan  3 18:19:32 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 09:19:32 -0800
Subject: [M3devel] AMD64_LINUX update...
Message-ID: <20140103171932.F28D61A2080@async.async.caltech.edu>


Hi again m3devel,

With the head and user threads, after bootstrapping via AMD64_FREEBSD,
I'm getting everything (apparently) working EXCEPT one thing:

stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
into an infinite loop, looks like it's happening on the stack because I
run out of memory rather quickly, on certain types local to my own system.

Has anyone seen this before?  I've never seen it on any 32-bit system
but I have a vague memory of seeing it on 64-bit systems.

Will debug over the weekend...

I'd be grateful for any input from the list on the pthreads issues, though.

     Mika


From mika at async.caltech.edu  Sat Jan  4 19:55:11 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 10:55:11 -0800
Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX
Message-ID: <20140104185511.9000A1A2080@async.async.caltech.edu>


I have this program that generates Modula-3 code from XML.

It uses Wr.PutText a lot, on small strings.

It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
Core i7 4960X.

(176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w

(29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w

I believe the issue is that Text.m3 is making a ton of system calls!

Here's a typical traceback:

(gdb) where
#0  0x00002aaaaaacb770 in ?? ()
#1  0x00002aaaaaacba1b in gettimeofday ()
#2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
#3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
#4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
#5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
#6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>, 
    M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
#7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
#8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
#9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>, 
    M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
#10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
#11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
#12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>, 
    M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
#13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
#14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>, 
    M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>, 
    M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>, 
    M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
#15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
#16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
(gdb) 

I believe all the TextStats operations are killing the performance.  Can we remove them?

Here is SetChars:

PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
  BEGIN
    TextStats.NoteGround (TextStats.Op.SetChars);
    TextStats.NoteGround (TextStats.Op.get_chars);
    t.get_chars (a, start);
    TextStats.NoteFinished (TextStats.Op.get_chars);
    TextStats.NoteFinished (TextStats.Op.SetChars);
  END SetChars;

I'm not sure how best to do this.  Removing them without killing functionality might not be
easy.  Looks like a case for conditional compilation.

Maybe generate the .m3s from some other source, with and without the TextStats, depending on
a setting in m3makefile?

Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.  

     Mika



From dragisha at m3w.org  Sat Jan  4 22:04:22 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:04:22 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140103171932.F28D61A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
Message-ID: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>

Hi Mika,

I have this for LINUXLIBC6, and I am using it for remake of HEAD: cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz

Can you build and share cm3-bin-core for AMD64_LINUX?

TIA,
dd

On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:

> 
> Hi again m3devel,
> 
> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
> I'm getting everything (apparently) working EXCEPT one thing:
> 
> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
> into an infinite loop, looks like it's happening on the stack because I
> run out of memory rather quickly, on certain types local to my own system.
> 
> Has anyone seen this before?  I've never seen it on any 32-bit system
> but I have a vague memory of seeing it on 64-bit systems.
> 
> Will debug over the weekend...
> 
> I'd be grateful for any input from the list on the pthreads issues, though.
> 
>     Mika

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment-0001.sig>

From mika at async.caltech.edu  Sat Jan  4 22:12:00 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:12:00 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
Message-ID: <20140104211200.A80571A2080@async.async.caltech.edu>

Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?

I can do this for both AMD64_LINUX (user threads)
and ARM_LINUX (pthreads)

I might still have AMD64_LINUX (pthreads)

Also have AMD64_FREEBSD (pthreads) from the head.

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=us-ascii
>
>Hi Mika,
>
>I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>
>Can you build and share cm3-bin-core for AMD64_LINUX?
>
>TIA,
>dd
>
>On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>
>>=20
>> Hi again m3devel,
>>=20
>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>> I'm getting everything (apparently) working EXCEPT one thing:
>>=20
>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>> into an infinite loop, looks like it's happening on the stack because =
>I
>> run out of memory rather quickly, on certain types local to my own =
>system.
>>=20
>> Has anyone seen this before?  I've never seen it on any 32-bit system
>> but I have a vague memory of seeing it on 64-bit systems.
>>=20
>> Will debug over the weekend...
>>=20
>> I'd be grateful for any input from the list on the pthreads issues, =
>though.
>>=20
>>     Mika
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/html;
>	charset=us-ascii
>
><html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; font-family: Candara; =
>border-spacing: 0px;"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>space; -webkit-line-break: after-white-space; ">dd</div><div =
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>-webkit-line-break: after-white-space; =
>"><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>at 18:19, <a =
>href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>infinite loop, looks like it's happening on the stack because I<br>run =
>out of memory rather quickly, on certain types local to my own =
>system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>for any input from the list on the pthreads issues, though.<br><br> =
>&nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>=
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Transfer-Encoding: 7bit
>Content-Disposition: attachment;
>	filename=signature.asc
>Content-Type: application/pgp-signature;
>	name=signature.asc
>Content-Description: Message signed with OpenPGP using GPGMail
>
>-----BEGIN PGP SIGNATURE-----
>Comment: GPGTools - http://gpgtools.org
>
>iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>=sMKJ
>-----END PGP SIGNATURE-----
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--


From mika at async.caltech.edu  Sat Jan  4 22:15:51 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:15:51 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
In-Reply-To: <EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
References: <20140103161343.F25291A2080@async.async.caltech.edu>
	<EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
Message-ID: <20140104211551.54F481A2080@async.async.caltech.edu>

Antony Hosking writes:
>I wonder of the redzone size for AMD 64 stack frames is not set properly.=20=
>

Tony,

Are you talking about the difference between the ARM and AMD64 results?

The AMD64 is also a machine with 12 logical (HT) processors whereas the
ARM is a single processor.  That could make a difference in how it fails.

By the way, would you know:  The following output on the amd64:

Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Is that an assertion failing in the pthreads library?  My program isn't
printing this stuff: it's coming from ThreadPThread or some such code.

     Mika

>
>Sent from my iPhone
>
>> On Jan 3, 2014, at 10:13 AM, mika at async.caltech.edu wrote:
>>=20
>> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LIN=
>UX/threadtest
>> GNU gdb (GDB) 7.4.1-debian
>> Copyright (C) 2012 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.ht=
>ml>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"=
>
>> and "show warranty" for details.
>> This GDB was configured as "arm-linux-gnueabihf".
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>...
>> Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/th=
>read/ARM_LINUX/threadtest...done.
>> (gdb) handle SIGILL=20
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        Yes       Yes     Yes             Illegal instruction
>> (gdb) handle SIGILL  nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        No        No      Yes             Illegal instruction
>> (gdb) run
>> Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/threa=
>d/ARM_LINUX/threadtest=20
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.=
>1".
>> Writing file...done
>> Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
>> [New Thread 0x40bf4450 (LWP 2942)]
>> [New Thread 0x40df4450 (LWP 2943)]
>> done
>> Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
>> [New Thread 0x411f4450 (LWP 2945)]
>> [New Thread 0x413f4450 (LWP 2946)]
>> done
>> Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
>> [New Thread 0x417f4450 (LWP 2948)]
>> [New Thread 0x419f4450 (LWP 2949)]
>> done
>> Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
>> [New Thread 0x41df4450 (LWP 2951)]
>> [New Thread 0x41ff4450 (LWP 2952)]
>> done
>> running...printing oldest/median age/newest
>>=20
>> Program received signal SIG64, Real-time event 64.
>> [Switching to Thread 0x41ff4450 (LWP 2952)]
>> 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnu=
>eabihf/libpthread.so.0
>> (gdb) handle SIG64 nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIG64         No        No      Yes             Real-time event 64
>> (gdb) cont
>> Continuing.
>> .
>>=20
>> ***
>> *** runtime error:
>> ***    An array subscript was out of range.
>> ***    file "../src/runtime/common/RTCollector.m3", line 418
>> ***
>>=20
>>=20
>> Program received signal SIGABRT, Aborted.
>> [Switching to Thread 0x40ff4450 (LWP 2944)]
>> 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> (gdb) where
>> #0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> #1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
>> #2  0x00000000 in ?? ()
>> (gdb)=20
>>=20
>> It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
>> ARM_LINUX but would be surprised if it didn't work.
>>=20
>> Yes the pthreads seem "more reliable" than they used to be, but I would
>> never use them in any sort of mission-critical application.  I use
>> Modula-3 programs for (among other things) financial transactions and
>> real-time motor controllers...
>>=20
>>    Mika


From dragisha at m3w.org  Sat Jan  4 22:52:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:52:32 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104211200.A80571A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
Message-ID: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>

-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*

there they are. I don?t do this too often, but both do-cm3-core and do-cm3-min will do for what I need.

TIA,
dd

On 04 Jan 2014, at 22:12, mika at async.caltech.edu wrote:

> Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?
> 
> I can do this for both AMD64_LINUX (user threads)
> and ARM_LINUX (pthreads)
> 
> I might still have AMD64_LINUX (pthreads)
> 
> Also have AMD64_FREEBSD (pthreads) from the head.
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=us-ascii
>> 
>> Hi Mika,
>> 
>> I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>> cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>> 
>> Can you build and share cm3-bin-core for AMD64_LINUX?
>> 
>> TIA,
>> dd
>> 
>> On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> Hi again m3devel,
>>> =20
>>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>>> I'm getting everything (apparently) working EXCEPT one thing:
>>> =20
>>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>>> into an infinite loop, looks like it's happening on the stack because =
>> I
>>> run out of memory rather quickly, on certain types local to my own =
>> system.
>>> =20
>>> Has anyone seen this before?  I've never seen it on any 32-bit system
>>> but I have a vague memory of seeing it on 64-bit systems.
>>> =20
>>> Will debug over the weekend...
>>> =20
>>> I'd be grateful for any input from the list on the pthreads issues, =
>> though.
>>> =20
>>>    Mika
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/html;
>> 	charset=us-ascii
>> 
>> <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>> charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>> Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>> for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>> 8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>> orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>> share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>> apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; font-family: Candara; =
>> border-spacing: 0px;"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>> ">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>> space; -webkit-line-break: after-white-space; ">dd</div><div =
>> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>> -webkit-line-break: after-white-space; =
>> "><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>> at 18:19, <a =
>> href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>> wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>> type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>> threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>> everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>> derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>> infinite loop, looks like it's happening on the stack because I<br>run =
>> out of memory rather quickly, on certain types local to my own =
>> system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>> any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>> systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>> for any input from the list on the pthreads issues, though.<br><br> =
>> &nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>> =
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Transfer-Encoding: 7bit
>> Content-Disposition: attachment;
>> 	filename=signature.asc
>> Content-Type: application/pgp-signature;
>> 	name=signature.asc
>> Content-Description: Message signed with OpenPGP using GPGMail
>> 
>> -----BEGIN PGP SIGNATURE-----
>> Comment: GPGTools - http://gpgtools.org
>> 
>> iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>> taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>> zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>> 0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>> hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>> 1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>> =sMKJ
>> -----END PGP SIGNATURE-----
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment-0001.sig>

From mika at async.caltech.edu  Sat Jan  4 22:58:53 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:58:53 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
Message-ID: <20140104215853.09F0D1A2080@async.async.caltech.edu>

Hi dd,

I already have everything built.  

What I'm asking is how I go from what I have to a tar file.  What is
included?  Is there a script for that?

I can just give you the full binary directory or the full cm3 system with
sources, but I suspect there is a reduced set you would prefer?

BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
scripts in python work fine.  If you already have cm3 from the head
it ought to be possible to cross-build.  But I did have problems going from
32 to 64 bits...

     Mika

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>
>
>--Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=windows-1252
>
>-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>
>there they are. I don=92t do this too often, but both do-cm3-core and =
>do-cm3-min will do for what I need.
>
>TIA,
>dd


From dragisha at m3w.org  Sat Jan  4 23:10:04 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:10:04 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104215853.09F0D1A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
Message-ID: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>

do-cm3-bin will build a tar.gz, I think. And it is not too big.

I did not try boot*.py as this box I would like to build for is remote and all? I have src.rpm and I only need this cm3-min? to make it work and build itself on remote box.

TIA,
dd

On 04 Jan 2014, at 22:58, mika at async.caltech.edu wrote:

> Hi dd,
> 
> I already have everything built.  
> 
> What I'm asking is how I go from what I have to a tar file.  What is
> included?  Is there a script for that?
> 
> I can just give you the full binary directory or the full cm3 system with
> sources, but I suspect there is a reduced set you would prefer?
> 
> BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
> scripts in python work fine.  If you already have cm3 from the head
> it ought to be possible to cross-build.  But I did have problems going from
> 32 to 64 bits...
> 
>     Mika
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>> 
>> 
>> --Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=windows-1252
>> 
>> -rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>> -rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>> -rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>> -rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>> -rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>> -rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>> -rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>> 
>> there they are. I don=92t do this too often, but both do-cm3-core and =
>> do-cm3-min will do for what I need.
>> 
>> TIA,
>> dd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment-0001.sig>

From mika at async.caltech.edu  Sat Jan  4 23:23:21 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:23:21 -0800
Subject: [M3devel] can't write to repository
Message-ID: <20140104222321.B53051A2080@async.async.caltech.edu>

Hi m3devel,

I can't seem to write to the CVS repository.  I get:

(62)rover:~/cm3-writable/cm3/scripts>cvs up -d .
mika at birch.elegosoft.com's password: 

Has my account been removed?  I don't think I've changed keys.

In any case, could someone please commit the following change:

~/cm3-cvs-anon/cm3/scripts>cvs diff sysinfo.sh
Index: sysinfo.sh
===================================================================
RCS file: /usr/cvs/cm3/scripts/sysinfo.sh,v
retrieving revision 1.109
diff -r1.109 sysinfo.sh
273a274
>         arm*)    CM3_TARGET=ARM_LINUX;;

    Mika


From mika at async.caltech.edu  Sat Jan  4 23:48:13 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:48:13 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
Message-ID: <20140104224813.34D171A2080@async.async.caltech.edu>


I think the script you meant was make-bin-dist-min.sh

The results are at:

http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz

Please upload this to the CM3 website if it looks OK to you and you know how to upload it.

It's set up to use user threads.

    Mika



From dragisha at m3w.org  Sat Jan  4 23:57:55 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:57:55 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104224813.34D171A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
Message-ID: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>

I am using it to bootstrap my rpm, did you try copying to: /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

I will make my rpm tomorrow, and try uploading all of it then.

Thanks,
dd



On 04 Jan 2014, at 23:48, mika at async.caltech.edu wrote:

> 
> I think the script you meant was make-bin-dist-min.sh
> 
> The results are at:
> 
> http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz
> 
> Please upload this to the CM3 website if it looks OK to you and you know how to upload it.
> 
> It's set up to use user threads.
> 
>    Mika
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment-0001.sig>

From mika at async.caltech.edu  Sat Jan  4 23:59:24 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:59:24 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
Message-ID: <20140104225924.858A21A2080@async.async.caltech.edu>

>I am using it to bootstrap my rpm, did you try copying to: =
>/var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

Per my email of a few minutes ago, I am unable to get birch to accept
my login credentials...


From dragisha at m3w.org  Sun Jan  5 01:44:17 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 01:44:17 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104225924.858A21A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
Message-ID: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>

Copied, and did ?a sensible thing?, ran update script? It did not create as complete index.html as it was before. I will look into it tomorrow, it is too late for me now to believe my further fixing would not break more things :).


On 04 Jan 2014, at 23:59, mika at async.caltech.edu wrote:

>> I am using it to bootstrap my rpm, did you try copying to: =
>> /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?
> 
> Per my email of a few minutes ago, I am unable to get birch to accept
> my login credentials...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/841337b6/attachment-0001.sig>

From mika at async.caltech.edu  Sun Jan  5 03:50:44 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 18:50:44 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
Message-ID: <20140105025044.9FC551A2080@async.async.caltech.edu>


Another favor?

Could you upload to CM3 the file at

http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz 

If there is a "notes" section, you can note that the distribution works on 

1.  Raspberry Pi running Raspbian Wheezy
Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l

2.  BeagleBone Black running (BeagleBoard)Ubuntu 13.10 (GNU/Linux 3.8.13-bone28 armv7l)
Linux arm 3.8.13-bone28 #1 SMP Fri Sep 13 01:11:14 UTC 2013 armv7l armv7l armv7l GNU/Linux

(Anyone reading this mail who wants a copy can of course also use the link to rover above.)

     Mika


From dragisha at m3w.org  Sun Jan  5 21:15:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 21:15:23 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140105025044.9FC551A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
Message-ID: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>

Also on http://opencm3.net/snaps/


On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:

> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment-0001.sig>

From dragisha at m3w.org  Mon Jan  6 06:02:36 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 6 Jan 2014 06:02:36 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
	<6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
Message-ID: <9C223752-0979-4BC0-84C7-C8F83B9BF01B@m3w.org>

There is hourly run script for user m3 which updates index file in this folder. It keeps deleteng your AMD_LINUX snapshot, apparently after indexing it.

ARM_LINUX one is indexed and left in peace. I will look into this issue in few days.

dd

On 05 Jan 2014, at 21:15, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Also on http://opencm3.net/snaps/
> 
> 
> On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:
> 
>> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment-0001.sig>

From rodney_bates at lcwb.coop  Mon Jan  6 20:41:53 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 06 Jan 2014 13:41:53 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CB0701.9030307@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.
>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.

Try the fix I just checked in to TextClass.m3.  It was already set up with such a boolean flag
(TextClass.CollectStats, set FALSE), but in one of the instrumentation procedures, two system
calls were outside the test thereof.  Hopefully, this will give a good bit of relief for now,
with a simple fix.

>
>       Mika
>
>



From rodney_bates at lcwb.coop  Thu Jan  9 22:01:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 09 Jan 2014 15:01:55 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CF0E43.2030701@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.

Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
only needed for performance study of the algorithms, which likely won't be done often.

>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
>
>       Mika
>
>



From jay.krell at cornell.edu  Fri Jan 10 09:22:25 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 10 Jan 2014 08:22:25 +0000
Subject: [M3devel] performance problems in CM3 Text operations on
 AMD64_LINUX
In-Reply-To: <52CF0E43.2030701@lcwb.coop>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
Message-ID: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>

Ok either, way, but here is another point.

In some of our code we do:

CONST DEBUG = FALSE;
IF FALSE THEN   xxEND;


It will tend to be optimized.Furthermore, syscalls -- even if not optimized,IF FALSE is going to be way faster than a syscall.

Also, gettimeofday shouldn't necessarily be a syscall.

But/and there are/should-be faster ways of getting time for benchmarking.

On Windows, GetTickCount and GetTickCount64 just read a value out of memory.Even GetSystemTimeAsFileTime is just a read of a 64bit value from memory.(The memory is read only, at the same address in all processes, and updated bythe kernel.)x86 has rdtsc, a 64bit hardware cycle counter readable in one instruction.



We should consider researching analogs to rdtsc across mips/powerpc/sparc/arm andprovide something in m3core. I think.Add it to the backend interface likely, for inlining.or maybe it is worth a function call, then just use #ifdefed C.


Maybe Posix provides something that tends to be like this?

Mika, how did you find this?Control-c every so often and look at callstack?Or a lot of stepping?

 - Jay

: Thu, 9 Jan 2014 15:01:55 -0600
> From: rodney_bates at lcwb.coop
> To: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] performance problems in CM3 Text operations on	AMD64_LINUX
> 
> 
> 
> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> > I have this program that generates Modula-3 code from XML.
> >
> > It uses Wr.PutText a lot, on small strings.
> >
> > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> > Core i7 4960X.
> >
> > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
> >
> > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
> >
> > I believe the issue is that Text.m3 is making a ton of system calls!
> >
> > Here's a typical traceback:
> >
> > (gdb) where
> > #0  0x00002aaaaaacb770 in ?? ()
> > #1  0x00002aaaaaacba1b in gettimeofday ()
> > #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> > #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> > #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> > #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> > #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
> >      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> > #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> > #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> > #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
> >      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
> >      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
> >      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
> >      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
> >      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> > (gdb)
> >
> > I believe all the TextStats operations are killing the performance.  Can we remove them?
> >
> > Here is SetChars:
> >
> > PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
> >    BEGIN
> >      TextStats.NoteGround (TextStats.Op.SetChars);
> >      TextStats.NoteGround (TextStats.Op.get_chars);
> >      t.get_chars (a, start);
> >      TextStats.NoteFinished (TextStats.Op.get_chars);
> >      TextStats.NoteFinished (TextStats.Op.SetChars);
> >    END SetChars;
> >
> > I'm not sure how best to do this.  Removing them without killing functionality might not be
> > easy.  Looks like a case for conditional compilation.
> 
> Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
> by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
> only needed for performance study of the algorithms, which likely won't be done often.
> 
> >
> > Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> > a setting in m3makefile?
> >
> > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
> >
> >       Mika
> >
> >
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140110/fe64b4e8/attachment-0001.html>

From mika at async.caltech.edu  Fri Jan 10 16:27:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 10 Jan 2014 07:27:10 -0800
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
	<COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
Message-ID: <20140110152711.036F21A208F@async.async.caltech.edu>


>Mika=2C how did you find this?Control-c every so often and look at callstac=
>k?Or a lot of stepping?

"Poor man's profiling" -- since profiling is usually broken in Modula-3.  It worked back with SRC M3 but that was a while ago :-)

1. run program in gdb
2. close eyes
3. count to three
4. hit ctrl-C
5. open eyes

(repeat a few times)

"close eyes" is important!


From dragisha at m3w.org  Sun Jan 12 09:49:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 09:49:23 +0100
Subject: [M3devel] Conversion to Git
Message-ID: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>

Hi,

I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.

For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.

alexb
darko
dbenavid
dragisha
eiserlohpp
hosking
hudson
jkrell
khaeusler
kschleiser
m3
mand
micha
mika
mm
neels
pmckinna
rcoleburn
rforb
rillig
rodney
sk
stsp
thielema
uamoore
wagner
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment-0001.sig>

From mika at async.caltech.edu  Sun Jan 12 18:53:17 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:53:17 -0800
Subject: [M3devel] CM3 crashing
Message-ID: <20140112175317.9F0031A2080@async.async.caltech.edu>

Hi m3devel,

Anyone have an idea what's going on here?

I have program that compiles with PM3 (FreeBSD4):

...
new source -> compiling ../src/Main.m3
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

new opaque info -> recompiling HParseStd.m3
...

With CM3 at head (AMD64_LINUX (userthreads)), the following happens:

--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure


***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
***

Abort

(gdb) run -x
Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

Program received signal SIGSEGV, Segmentation fault.
0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
36            u.get_info (RInfo);  
(gdb) where
#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading variable>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variable>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:1289
#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading variable>) at ../src/misc/CG.m3:2547
#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable>) at ../src/values/Procedure.m3:358
#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) at ../src/values/Value.m3:61
#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variable>) at ../src/exprs/ProcExpr.m3:96
#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:145
#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading variable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error reading variable>) at ../src/values/Formal.m3:550
#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading variable>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:459
#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:131
#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:431
#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:91
#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading variable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker.m3:540
#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading variable>) at ../src/stmts/ReturnStmt.m3:55
#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:345
#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable>) at ../src/stmts/IfStmt.m3:105
#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:339
#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading variable>) at ../src/stmts/BlockStmt.m3:103
#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading variable>) at ../src/values/Procedure.m3:577
#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading variable>) at ../src/values/Procedure.m3:541
#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading variable>) at ../src/misc/ProcBody.m3:170
#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error reading variable>) at ../src/misc/ProcBody.m3:69
#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:905
#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:837
#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading variable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error reading variable>) at ../src/misc/M3Front.m3:66
#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variable>) at ../src/Builder.m3:1705
#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error reading variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m3:1025
#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_AicXUJ_shared=<error reading variable>, 
    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading variable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variable>) at ../src/QMachine.m3:1773
#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_file=<error reading variable>) at ../src/M3Build.m3:679
#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading variable>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../src/Main.m3:231
#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe390) at _m3main.c:22
(gdb) 

Looks like something very basic...





From mika at async.caltech.edu  Sun Jan 12 18:58:30 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:58:30 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175317.9F0031A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
Message-ID: <20140112175830.C51881A2080@async.async.caltech.edu>

Sigh...

    188 PROCEDURE PutErr (u: U;  a, b, c: TEXT := NIL) =
    189   BEGIN
    190     u.note_error ("********* M3CG_Check ERROR *********** " & a & b & c);
    191     u.child.comment ("********* M3CG_Check ERROR *********** ", a, b, c);
    192     INC (u.n_errors);
    193   END PutErr;

Ah.... ok....

"../src/Main.m3", line 129: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 131: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 133: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"

The code is:

              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
            |
              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
            |
              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
 
I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 

mika writes:
>Hi m3devel,
>
>Anyone have an idea what's going on here?
>
>I have program that compiles with PM3 (FreeBSD4):
>
>...
>new source -> compiling ../src/Main.m3
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>new opaque info -> recompiling HParseStd.m3
>...
>
>With CM3 at head (AMD64_LINUX (userthreads)), the following happens:
>
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>
>***
>*** runtime error:
>***    Attempt to reference an illegal memory location.
>***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
>***
>
>Abort
>
>(gdb) run -x
>Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
>[Thread debugging using libthread_db enabled]
>Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>Program received signal SIGSEGV, Segmentation fault.
>0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M
>3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>36            u.get_info (RInfo);  
>(gdb) where
>#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable
>>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading varia
>ble>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variabl
>e>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
>#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading
> variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
>#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error re
>ading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3
>:1289
>#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading varia
>ble>) at ../src/misc/CG.m3:2547
>#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable
>>) at ../src/values/Procedure.m3:358
>#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) a
>t ../src/values/Value.m3:61
>#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variab
>le>) at ../src/exprs/ProcExpr.m3:96
>#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>)
> at ../src/exprs/Expr.m3:145
>#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error readi
>ng variable>) at ../src/values/Formal.m3:550
>#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading varia
>ble>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error readi
>ng variable>) at ../src/values/Formal.m3:459
>#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:131
>#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:
>431
>#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:91
>#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading va
>riable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker
>.m3:540
>#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading vari
>able>) at ../src/stmts/ReturnStmt.m3:55
>#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:345
>#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable
>>) at ../src/stmts/IfStmt.m3:105
>#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:339
>#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading varia
>ble>) at ../src/stmts/BlockStmt.m3:103
>#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading varia
>ble>) at ../src/values/Procedure.m3:577
>#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading vari
>able>) at ../src/values/Procedure.m3:541
>#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading varia
>ble>) at ../src/misc/ProcBody.m3:170
>#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error readin
>g variable>) at ../src/misc/ProcBody.m3:69
>#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading va
>riable>) at ../src/values/Module.m3:905
>#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable
>>) at ../src/values/Module.m3:837
>#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
>#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading var
>iable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error readi
>ng variable>) at ../src/misc/M3Front.m3:66
>#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>
>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variab
>le>) at ../src/Builder.m3:1705
>#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
>#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
>#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading vari
>able>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
>#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error readi
>ng variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m
>3:1025
>#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading
> variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<erro
>r reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
>    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
>#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading var
>iable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error re
>ading variable>, M3_AicXUJ_shared=<error reading variable>, 
>    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
>#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading va
>riable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
>#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading varia
>ble>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
>#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variab
>le>) at ../src/QMachine.m3:1773
>#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
>#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd
>56fi_file=<error reading variable>) at ../src/M3Build.m3:679
>#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading va
>riable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
>#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading varia
>ble>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
>#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3
>_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
>#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, 
>M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
>#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
>#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at
> ../src/Main.m3:231
>#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading va
>riable>) at ../src/runtime/common/RTLinker.m3:408
>#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading varia
>ble>) at ../src/runtime/common/RTLinker.m3:115
>#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variab
>le>) at ../src/runtime/common/RTLinker.m3:124
>#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe3
>90) at _m3main.c:22
>(gdb) 
>
>Looks like something very basic...
>
>


From hosking at cs.purdue.edu  Sun Jan 12 21:30:34 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 12 Jan 2014 15:30:34 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175830.C51881A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
Message-ID: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>

Are you saying passing these used to work in PM3?
Sounds like a front-end bug.  I?m curious what changed to break it.

On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

> 
> The code is:
> 
>              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
>            |
>              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
>            |
>              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
> 
> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484







From dragisha at m3w.org  Sun Jan 12 22:45:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 22:45:32 +0100
Subject: [M3devel] Conversion to Git
In-Reply-To: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
References: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
Message-ID: <E4D3225F-F1C5-4660-9EFB-6EA49218733F@m3w.org>

Thanks to everybody who send me infomation, esp. to Peter Eiserloh who gave me information + idea how to fill rest. Missing are emails from some Elego people and I expect I will have that soon.

I will keep you all posted on conversion status. Current version is on Github (dragisha/cm3), and I am checking how it matches with local CVS checkouts.

On 12 Jan 2014, at 09:49, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Hi,
> 
> I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.
> 
> For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.
> 
> alexb
> darko
> dbenavid
> dragisha
> eiserlohpp
> hosking
> hudson
> jkrell
> khaeusler
> kschleiser
> m3
> mand
> micha
> mika
> mm
> neels
> pmckinna
> rcoleburn
> rforb
> rillig
> rodney
> sk
> stsp
> thielema
> uamoore
> wagner
> --
> Dragi?a Duri?
> dragisha at m3w.org
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment-0001.sig>

From mika at async.caltech.edu  Mon Jan 13 01:25:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 16:25:39 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
Message-ID: <20140113002539.EC4EE1A2080@async.async.caltech.edu>

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>


From hosking at cs.purdue.edu  Mon Jan 13 16:46:36 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 10:46:36 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <09749A02-B9B9-4C74-8CB4-73379E91385B@cs.purdue.edu>


On Jan 12, 2014, at 7:25 PM, mika at async.caltech.edu wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

Interesting.  Needs exploration.  This should be filed as a bug.  The workaround is to wrap the builtin calls (Word.Xor, etc.) in a local procedure so that you have an addressable procedure.

> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?

Yes, they are inlined, but there *is* an implementation in Word.m3 (generated from GenWord.mg).

> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?

It must be the threads implementation, since the same collector is used with both pthreads and user threads, and I believe that user threads don?t suffer from this problem.  My intuition is that it may be a problem with getting stack roots on particular platforms, probably because the stack bounds are not being computed correctly for a given architecture, or perhaps because reference alignment is not being observed correctly.  For example, is it possible on x86_64 to have references stored in the stack aligned on 32-bit instead of 64-bit boundaries?

I notice that RTThread.i3 says the pointer alignment is BYTESIZE(ADDRESS) on all platforms.  Once upon a time the alignment was platform-dependent, so I wonder if we lost something somewhere with all the backend and porting work.  It may be that the backend needs to be instructed to align all references appropriately in the stacks.

One way to test all this might be to see if the thread stress tester causes things to break on older 32-bit platforms, where I expect all pointers will be aligned properly.

Also, I?ve not inspected the code in ThreadPThreadC.c recently (it?s changed since I last looked), but I wonder if the stack bounds are being computed correctly so that the collector can find all the stack roots.

I?d really like to get to the bottom of this problem.

> Tony Hosking writes:
>> Are you saying passing these used to work in PM3?
>> Sounds like a front-end bug.  I=92m curious what changed to break it.
>> 
>> On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> The code is:
>>> =20
>>>             HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Xor), ab)
>>>           |
>>>             HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Or), ab)
>>>           |
>>>             HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.And), ab)
>>> =20
>>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>> 
>> 
>> 
>> Antony Hosking | Associate Professor | Computer Science | Purdue =
>> University
>> 305 N. University Street | West Lafayette | IN 47907 | USA
>> Mobile +1 765 427 5484
>> 
>> 
>> 
>> 



From hosking at cs.purdue.edu  Mon Jan 13 17:03:38 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 11:03:38 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <EA2F5D60-CBC8-47A1-BC34-B3981DAFE4BD@cs.purdue.edu>

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:

> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 
> 
> Regards Peter
> 
> 
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/202c6d98/attachment-0001.html>

From rcolebur at SCIRES.COM  Mon Jan 13 20:16:41 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:16:41 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy

Randy C. Coleburn, CISSP-ISSEP, GCED
Corporate Information Security Officer (CISO)
Scientific Research Corporation
2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339
voice: (770) 989-9464, email: RColeburn at SciRes.com<mailto:RColeburn at SciRes.com>, fax: (770) 989-9497

[logo_SRC]
"Technology Driven-Customer Focused"

Quality Policy:  "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements."

CONFIDENTIALITY NOTICE:  This email constitutes an electronic communication within the meaning of the Electronic Communications Privacy Act, 18 U.S.C. 2510, and its disclosure is strictly limited to the named recipient(s) intended by the sender of this message.  This email, and any attachments, may contain confidential and/or proprietary information of Scientific Research Corporation.  If you are not a named recipient, any copying, using, disclosing or distributing to others the information in this email and attachments is STRICTLY PROHIBITED.  If you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts or hard copies of the email and attachments.

EXPORT COMPLIANCE NOTICE:  This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR).  Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer.  In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization.  By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:


Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 11339 bytes
Desc: image002.jpg
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment-0001.jpg>

From rcolebur at SCIRES.COM  Mon Jan 13 20:55:44 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:55:44 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/3dbd395c/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Jan 13 21:02:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 15:02:57 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <227E8C61-B070-49FF-8AD5-E23420E9C1B0@cs.purdue.edu>

Is this from the head of source?
Also, please run with @M3paranoidgc flag.


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 
>  
> C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386
>  
> C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2013-12-17 17:28:52
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> Here is the output:
>  
> C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> This last assert repeats indefinitely until you press CTRL-C.
>  
> --Randy
>  
>  
> From: Coleburn, Randy 
> Sent: Monday, January 13, 2014 2:17 PM
> To: Tony Hosking; Peter McKinna
> Cc: m3devel
> Subject: Re: [M3devel] CM3 crashing
>  
> Tony et al:
>  
> The thread test program fails for me on 64-bit Windows 7.
>  
> I?ve listed output from a couple of runs below:
>  
> C:\cm3\Sandbox>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2014-01-07 06:55:14
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> FIRST RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\runtime\common\RTCollector.m3", line 418
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
>                ?this last assertion repeats until you press CTRL-C to abort the program?
>  
> SECOND RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
> ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
> ....
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x776f22d2
> ***
>  
> Stack trace:
>    FP         PC      Procedure
> ---------  ---------  -------------------------------
> 0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
> 0xa6f9d0  0x776f22d2  <???>
> 0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
> 0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
> 0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fb90  0x76db336a  <???>
> 0xa6fbd0  0x7770bf32  <???>
> .........  .........  ... more frames ...
>  
> Maybe this info will be useful in tracking down the problem.
>  
> --Randy
>  
>  
> From: Tony Hosking [mailto:hosking at cs.purdue.edu] 
> Sent: Monday, January 13, 2014 11:04 AM
> To: Peter McKinna
> Cc: m3devel
> Subject: EXT:Re: [M3devel] CM3 crashing
>  
> Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
> In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.
>  
> As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?
>  
> If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.
>  
> On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:
>  
> 
> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
> 
> Regards Peter
>  
>  
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/c56a7e47/attachment-0001.html>

From rcolebur at SCIRES.COM  Mon Jan 13 21:13:55 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 20:13:55 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:


I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/6198676d/attachment-0001.html>

From rcolebur at SCIRES.COM  Mon Jan 13 22:02:52 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 21:02:52 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/eea82060/attachment-0001.html>

From jay.krell at cornell.edu  Tue Jan 14 00:22:12 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 13 Jan 2014 23:22:12 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>,
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W762DFE2FF1E39C9147F90EE6BC0@phx.gbl>

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: 
 
  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html   
 
 
 And disappointed. Cooperative suspend will fix this someday. 
 
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: hosking at cs.purdue.edu
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.



 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the
 finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:



 





Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much
 always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok
 to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the
 alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.



It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 





 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/45717d65/attachment-0001.html>

From rcolebur at SCIRES.COM  Tue Jan 14 01:11:07 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Tue, 14 Jan 2014 00:11:07 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?

Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Monday, January 13, 2014 6:22 PM
To: Coleburn, Randy; Tony
Cc: m3devel
Subject: EXT:RE: [M3devel] CM3 crashing

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html


 And disappointed. Cooperative suspend will fix this someday.


 - Jay

________________________________
From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
To: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
Subject: Re: [M3devel] CM3 crashing
Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/7dd888c2/attachment-0001.html>

From jay.krell at cornell.edu  Tue Jan 14 01:21:49 2014
From: jay.krell at cornell.edu (Jay K)
Date: Tue, 14 Jan 2014 00:21:49 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W14788FF4A8D081652DA79DE6BF0@phx.gbl>

I don't know. We can try writing a stress test for it. It looks like one is available.
Agreed -- work out the bugs in the native Windows and Posix systems.
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: jay.krell at cornell.edu; hosking at cs.purdue.edu
Date: Tue, 14 Jan 2014 00:11:07 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Jay:
 
The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?
 
Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Monday, January 13, 2014 6:22 PM

To: Coleburn, Randy; Tony

Cc: m3devel

Subject: EXT:RE: [M3devel] CM3 crashing


 

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:


 

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html  


 

 

 And disappointed. Cooperative suspend will fix this someday. 

 

 

 - Jay

 






From: 
rcolebur at SCIRES.COM

To: hosking at cs.purdue.edu

Date: Mon, 13 Jan 2014 21:02:52 +0000

CC: m3devel at elegosoft.com

Subject: Re: [M3devel] CM3 crashing

Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll
 commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.


 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive)
 to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user
 threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without
 keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:


 




Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it
 with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with
 code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the
 read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only
 way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.


It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 




 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 




 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/2693506b/attachment-0001.html>

From peter.mckinna at gmail.com  Mon Jan 13 04:54:08 2014
From: peter.mckinna at gmail.com (Peter McKinna)
Date: Mon, 13 Jan 2014 14:54:08 +1100
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>

Hey,

  I was trying to get a handle on that problem last year. The threadtest
program is really a stress tester of the collector/allocator with pthreads.
If you run it with just the tests read and alloc you pretty much always get
a crash. If you run them with paranoidgc it will crash in the heap checker.
Tony thought it was a clear cut  problem of the roots of some ref not being
found on a thread stack. I mucked around with code to get some output and
the stacks looked ok to me but I could be wrong. All stacks are checked
whilst the threads are blocked in a signal handler, the design of which
looks fine as far as I can tell. This test is characterised by some slow
threads (the read threads) and a bunch of fast threads (the alloc threads).
Even if you modify the test to have only one read thread the problem
occurs. I have had misgivings about mixing signals and threads having been
bitten many years ago, but really this is the only way the collector can
get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the
collector itself. One thing I noticed is that in
ThreadPThread__ProcessStopped  the second call to   p(context, ((char
*)context) + sizeof(ucontext_t)); according to the comment is to process
the registers. But the registers should already be on the stack and anyway
this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the
collector has always been at the heart of m3 programs.

Regards Peter



On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a
> reason to switch to CM3).
>
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
>
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
>
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/9bfaddc0/attachment-0001.html>

From jay.krell at cornell.edu  Wed Jan 15 06:01:16 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 05:01:16 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>,
	<20140112175830.C51881A2080@async.async.caltech.edu>,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>

Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/7fcd3a4f/attachment-0001.html>

From jay.krell at cornell.edu  Wed Jan 15 07:48:30 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 06:48:30 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>, ,
	<20140112175830.C51881A2080@async.async.caltech.edu>, ,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>, ,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>, ,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>,
	<COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
Message-ID: <COL130-W18EA0744A6E2E144FEB51AE6BE0@phx.gbl>

ok, I augmented the test a bit and commited it to CVS and I can see it fail now.
Does the test make sense to people?
 
Thanks,
 - Jay

 
From: jay.krell at cornell.edu
To: peter.mckinna at gmail.com; mika at async.caltech.edu
Date: Wed, 15 Jan 2014 05:01:16 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing




Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/cb7c9be3/attachment-0001.html>

From jay.krell at cornell.edu  Sat Jan 18 07:22:53 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 18 Jan 2014 06:22:53 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <COL130-W549E4FE51DBF493AF61101E6BB0@phx.gbl>

 This program also doesn't behave as expected, native, nothing to do with wow64. 
 Anyone else please confirm: 
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it  
   3) their not being met -- stuff gets printed 
 This is in the CVS repository, scratch/wow64stack/sync2.cpp 
 I am following up further. 
 Maybe we should get cooperative suspend really going? 
Thank you.
 - Jay 
 
#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value; 
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/26ed6280/attachment-0001.html>

From rcolebur at SCIRES.COM  Sat Jan 18 08:11:45 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Sat, 18 Jan 2014 07:11:45 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

I'm very concerned about the threading not working properly on both 32-bit and 64-bit Windows.

The Thread Test program crashes for me on both platforms.

I haven't tried your new test program yet.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Saturday, January 18, 2014 1:23 AM
To: m3devel
Subject: EXT:[M3devel] Win32 SuspendThread?

 This program also doesn't behave as expected, native, nothing to do with wow64.
 Anyone else please confirm:
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it
   3) their not being met -- stuff gets printed
 This is in the CVS repository, scratch/wow64stack/sync2.cpp
 I am following up further.
 Maybe we should get cooperative suspend really going?
Thank you.
 - Jay

#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value;
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/ee809938/attachment-0001.html>

From dragisha at m3w.org  Sat Jan 18 09:24:11 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 18 Jan 2014 09:24:11 +0100
Subject: [M3devel] Cooperative suspend?
Message-ID: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>

Jay,

What exactly do you mean? Isn?t it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?

dd
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment-0001.sig>

From mika at async.caltech.edu  Sun Jan 19 04:40:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 18 Jan 2014 19:40:06 -0800
Subject: [M3devel] how is this possible?
Message-ID: <20140119034006.34F511A208F@async.async.caltech.edu>


Any hypothesis is welcome...

(311)truffles:~/t/btc/hparse/src>../AMD64_LINUX/hparse and.scm 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../AMD64_LINUX/BDDSchemeStubs.m3", line 619
***

Abort
(312)truffles:~/t/btc/hparse/src>

    613 PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } =
    614   BEGIN
    615     IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END;
    616     IF NOT ISTYPE(x,BDD.T) THEN
    617       RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
    618     END;
    619     RETURN x
    620   END ToModula_BDD_T;


The object is of type BDD.Root (I checked the TYPECODE), which is
declared thus:


TYPE
  Root = T BRANDED Brand & " Root" OBJECT
    mu : MUTEX; (* as yet unused *)
    id : CARDINAL;
    tab : BDDTripleHash.T;
    cache : ARRAY Op OF BDDTripleHash.T;
  END;

SchemeObject.T is the same as REFANY

I'm not even sure where to start debugging this.  The code "should work"...

Indeed:

> (rttype-typecode a)
144
> (RTBrand.GetByTC 144)
"BDD 0.1 Root"
> (RTBrand.GetByTC 143)
"BDD 0.1"
> (RTType.Supertype 143)
4

(4 is REFANY)

> (BDD.And a a)
EXCEPTION! RuntimeError! An explicit or implicit NARROW() operation failed.
> a-and-a
<Modula-3 object : BDDImpl.Root, BRANDED BDD 0.1 Root>
> (rttype-typecode a-and-a)
144


     Mika




From jay.krell at cornell.edu  Sun Jan 19 02:02:56 2014
From: jay.krell at cornell.edu (Jay)
Date: Sat, 18 Jan 2014 17:02:56 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>

For native I'm less worried now. For wow64 I'm still worried & hope to follow up further.

 - Jay

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:

> Jay:
>  
> I?m very concerned about the threading not working properly on both 32-bit and 64-bit Windows.
>  
> The Thread Test program crashes for me on both platforms.
>  
> I haven?t tried your new test program yet.
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Saturday, January 18, 2014 1:23 AM
> To: m3devel
> Subject: EXT:[M3devel] Win32 SuspendThread?
>  
>  This program also doesn't behave as expected, native, nothing to do with wow64. 
>  Anyone else please confirm: 
>    1) my expectations -- it should never print anything
>    2) their importance  -- garbage collector depends on it  
>    3) their not being met -- stuff gets printed 
>  This is in the CVS repository, scratch/wow64stack/sync2.cpp 
>  I am following up further. 
>  Maybe we should get cooperative suspend really going? 
> Thank you.
>  - Jay 
>  
> #include <stdio.h>
> #include <windows.h>
> volatile long value;
> unsigned long __stdcall Thread(PVOID parameter)
> {
>   while (1)
>    InterlockedIncrement(&value);
>   return 0;
> }
> int __cdecl main()
> {
>   HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
>   UINT i = 0;
>   while (1)
>   {
>     i += 1;
>     if (SuspendThread(thread) == (DWORD)-1)
>     {
>       printf("suspend failed %X\n", GetLastError());
>       Sleep(1);
>       continue;
>     }
>     volatile long a = value; 
>     volatile long b = value;
>     if (a != b)
>     {
>      printf("%d %d %d %d\n", i, a, b, b - a);
>     }
>     ResumeThread(thread);
>   }
> }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/21485aaa/attachment-0001.html>

From mika at async.caltech.edu  Sun Jan 19 10:41:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 01:41:39 -0800
Subject: [M3devel] something wrong with ISTYPE ...
Message-ID: <20140119094139.3917E1A208F@async.async.caltech.edu>


Hi m3devel,

There is definitely something wrong with ISTYPE.  The problem I reported earlier goes away if I change the code like so:

(416)truffles:~/t/btc/hparse/AMD64_LINUX>diff -c BDDSchemeStubs.m3.old BDDSchemeStubs.m3
*** BDDSchemeStubs.m3.old       2014-01-19 01:13:33.000000000 -0800
--- BDDSchemeStubs.m3   2014-01-19 01:36:52.000000000 -0800
***************
*** 613,622 ****
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     IF NOT ISTYPE(x,BDD.T) THEN
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END;
!     RETURN x
    END ToModula_BDD_T;
  
  
--- 613,624 ----
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     TYPECASE x OF
!       BDD.T(xx) => RETURN xx
!     ELSE
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END
! 
    END ToModula_BDD_T;

This shouldn't be possible... it's the same object as before, with the
same return type.  I don't get an exception (I shouldn't either)---the
return just behaves normally and the program works.  Why doesn't it work
with ISTYPE but only with TYPECASE?

BTW, this is an old bug.  Same behavior with ancient PM3.  Where to look
in the code?

    Mika


From adacore at marino.st  Sun Jan 19 11:03:27 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 11:03:27 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Message-ID: <52DBA2EF.4020101@marino.st>

Hi all,

I've been trying to port cm3 to DragonFly x86-64.
I'm able to build cm3cg on DragonFly, and use it to build libraries
(e.g. libm3core) that were targeted for AMD64_FREEBSD.

However, when I set the HOST to AMD64_FREEBSD and TARGET to
AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
target machine: TARGET = AMD64_DRAGONFLY" (context below)

=========================================================
> /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> ROOT        = /work/lang/modula3/work/modula3-5.8.6
> CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> M3GDB       = yes
> M3OSTYPE    = POSIX
> TARGET      = AMD64_DRAGONFLY
> GCC_BACKEND = yes
> INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> GREP        = egrep
> GMAKE       = gmake
> TMPDIR      = /var/tmp
> EXE         = 
> SL          = /
> TAR         = tar
> CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> CM3VERSION  = 5.8.6
> CM3VERSIONNUM = 050806
> CM3LASTCHANGED = 2010-04-11
> FIND = /usr/bin/find
> EGREP = egrep
> === package m3-libs/m3core ===
>  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> --- building in AMD64_DRAGONFLY ---
> 
> ignoring ../src/m3overrides
> 
> 
> Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> 
>  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> *** [assembler-code] Error code 1

=========================================================

I thought I added all the target definitions but apparently I've missed
something.  I also realize the current repo looks a lot different, but I
wanted to use the last official release.

I've attached a concatenated file of patches showing what I've changed.
 Could somebody suggest what data is missing that causes the error above?

Regards,
John
-------------- next part --------------
--- m3-libs/libm3/src/os/POSIX/m3makefile.orig	2009-07-24 10:02:40.000000000 +0000
+++ m3-libs/libm3/src/os/POSIX/m3makefile
@@ -84,6 +84,7 @@ local readonly DefaultOSName = {
    "MIPS64_OPENBSD" : "OpenBSD",
    "SPARC64_SOLARIS" : "Solaris",
    "I386_OPENBSD" : "OpenBSD",
+   "AMD64_DRAGONFLY" : "DragonFly",
    "AMD64_FREEBSD" : "FreeBSD",
    "PA32_HPUX" : "HPUX",
    "PA64_HPUX" : "HPUX",
@@ -92,6 +93,7 @@ local readonly DefaultOSName = {
    "AIX" : "AIX",
    "CYGWIN" : "Cygwin",
    "DARWIN" : "Darwin",
+   "DRAGONFLY" : "DragonFly",
    "FREEBSD" : "FreeBSD",
    "HPUX" : "HPUX",
    "INTERIX" : "Interix",
@@ -156,6 +158,7 @@ local readonly DefaultArch = {
    "SPARC64_SOLARIS" : "sparc64",
    "I386_OPENBSD" : "i686",
    "AMD64_FREEBSD" : "AMD64",
+   "AMD64_DRAGONFLY" : "AMD64",
    "PA32_HPUX" : "hppa",
    "PA64_HPUX" : "hppa64",
 
--- m3-libs/m3core/src/Csupport/m3makefile.orig	2009-07-24 10:51:01.000000000 +0000
+++ m3-libs/m3core/src/Csupport/m3makefile
@@ -11,6 +11,7 @@ readonly _CSupportPieces = {
 % "ALPHA_OSF"  : [ "little-endian" ],
 % "AIX386"     : [ "little-endian" ],
   "AMD64_DARWIN": [ "little-endian" ],
+  "AMD64_DRAGONFLY": [ "little-endian" ],
   "AMD64_FREEBSD": [ "little-endian" ],
   "AMD64_OPENBSD": [ "little-endian" ],
   "AMD64_NETBSD": [ "little-endian" ],
--- m3-libs/m3core/src/float/m3makefile.orig	2009-07-24 10:51:24.000000000 +0000
+++ m3-libs/m3core/src/float/m3makefile
@@ -13,6 +13,7 @@ _FloatPieces = {
 % "ALPHA_OSF"  : _float_le,
 % "AIX386"     : _float_le,
   "AMD64_DARWIN": _float_le,
+  "AMD64_DRAGONFLY": _float_le,
   "AMD64_FREEBSD": _float_le,
   "AMD64_OPENBSD": _float_le,
   "AMD64_NETBSD": _float_le,
--- m3-libs/m3core/src/m3core.h.orig	2010-03-28 10:07:45.000000000 +0000
+++ m3-libs/m3core/src/m3core.h
@@ -207,7 +207,8 @@ typedef INTEGER m3_uid_t;
 #define PTHREAD_TO_M3(x)   ((m3_pthread_t)(size_t)(x))
 #define PTHREAD_FROM_M3(x) ((pthread_t)(size_t)(x))
 
-#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__)
+#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__OpenBSD__) || defined(__DragonFly__)
 #define HAS_STAT_FLAGS
 #endif
 
--- m3-libs/m3core/src/platforms.quake.orig	2010-03-28 10:09:07.000000000 +0000
+++ m3-libs/m3core/src/platforms.quake
@@ -1,5 +1,6 @@
 _TARGET_OS = {
   "AMD64_DARWIN"  : "DARWIN",
+  "AMD64_DRAGONFLY" : "DRAGONFLY",
   "AMD64_FREEBSD" : "FREEBSD",
   "AMD64_LINUX"   : "LINUX",
   "AMD64_NETBSD"  : "NETBSD",
@@ -32,6 +33,7 @@ _TARGET_ENDIAN = {
 % This is not computed based on architecture
 % because some architectures vary, e.g. PPC_NT is little endian, PPC_DARWIN is big.
   "AMD64_DARWIN"    : "LITTLE",
+  "AMD64_DRAGONFLY" : "LITTLE",
   "AMD64_FREEBSD"   : "LITTLE",
   "AMD64_LINUX"     : "LITTLE",
   "AMD64_NETBSD"    : "LITTLE",
--- m3-libs/m3core/src/runtime/POSIX/RTSignalC.c.orig	2010-04-16 12:20:56.000000000 +0000
+++ m3-libs/m3core/src/runtime/POSIX/RTSignalC.c
@@ -167,7 +167,7 @@ static size_t GetPC(void* xcontext)
 #elif defined(__NetBSD__)
     _UC_MACHINE_PC(context)
 
-#elif defined(__FreeBSD__)
+#elif defined(__FreeBSD__) || defined(__DragonFly__)
 #if defined(__amd64)
     context->uc_mcontext.mc_rip
 #elif defined(__i386)
--- m3-libs/m3core/src/runtime/common/Compiler.tmpl.orig	2009-07-13 09:09:35.000000000 +0000
+++ m3-libs/m3core/src/runtime/common/Compiler.tmpl
@@ -21,7 +21,7 @@ readonly Compiler_i3 = [
   "               SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD,",
   "               SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD,",
   "               PA32_HPUX, PA64_HPUX, ARM_DARWIN, I386_INTERIX,",
-  "               AMD64_NETBSD, AMD64_OPENBSD };",
+  "               AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY };",
   "",
   "CONST",
   "  ThisOS       = OS." & OS_TYPE & ";",
--- m3-libs/m3core/src/thread.quake.orig	2009-12-20 11:01:02.000000000 +0000
+++ m3-libs/m3core/src/thread.quake
@@ -12,6 +12,7 @@ readonly NO_USER_THREADS =
     "AMD64_FREEBSD":1,
     "AMD64_OPENBSD":1,
     "AMD64_NETBSD":1,
+    "AMD64_DRAGONFLY":1,
     "AMD64_LINUX":1,
     "ARM_LINUX":1,
     "MIPS32_IRIX":1,
--- m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c.orig	2009-12-21 05:35:34.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c
@@ -2,7 +2,7 @@
 /* All rights reserved.                                            */
 /* See the file COPYRIGHT-PURDUE for a full description.           */
 
-#ifndef __FreeBSD__
+#if !defined(__FreeBSD__) && !defined(__DragonFly__)
 
 /* avoid empty file */
 
--- m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c.orig	2009-12-12 14:48:10.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c
@@ -23,7 +23,7 @@
 #endif
 #endif
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 #define M3_DIRECT_SUSPEND
 #endif
 
@@ -177,7 +177,7 @@ void ThreadPThread__sem_post(void)      
 void ThreadPThread__sem_getvalue(void)  { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 void ThreadPThread__sigsuspend(void)    { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 
 int
 ThreadPThread__SuspendThread (m3_pthread_t mt)
--- m3-libs/m3core/src/time/POSIX/m3makefile.orig	2009-07-24 07:26:15.000000000 +0000
+++ m3-libs/m3core/src/time/POSIX/m3makefile
@@ -11,6 +11,7 @@ readonly _DateImpls = {
 % "ALPHA_OSF"  : "DateBsd",
 % "AIX386"     : "DateBsd",
   "AMD64_DARWIN": "DateBsd",
+  "AMD64_DRAGONFLY": "DateBsd",
   "AMD64_FREEBSD": "DateBsd",
   "AMD64_OPENBSD": "DateBsd",
   "AMD64_NETBSD": "DateBsd",
@@ -63,6 +64,7 @@ readonly _DateImpls = {
 % "AIX"     : "DateBsd",
   "CYGWIN"  : "DatePosix",
   "DARWIN"  : "DateBsd",
+  "DRAGONFLY" : "DataBsd",
   "FREEBSD" : "DateBsd",
   "HPUX"    : "DatePosix",
   "INTERIX" : "DatePosix",
--- m3-libs/m3core/src/unix/Common/UtimeC.c.orig	2009-12-11 11:18:34.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/UtimeC.c
@@ -56,7 +56,8 @@ struct timeval
 #endif
 }
 
-#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__APPLE__)
+#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__APPLE__) || defined(__DragonFly__)
 #define M3BSD
 #endif
 
--- m3-libs/m3core/src/unix/Common/Uucontext.c.orig	2009-06-29 19:20:40.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/Uucontext.c
@@ -29,6 +29,7 @@ see http://www.opengroup.org/onlinepubs/
     || defined(__linux) \
     || defined(__NetBSD__) \
     || defined(__hpux) \
+    || defined(__DragonFly__) \
     || defined(__FreeBSD__) \
 
 int Uucontext__getcontext(ucontext_t* context)
--- m3-libs/m3core/src/unix/Common/m3makefile.orig	2009-07-19 00:28:17.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/m3makefile
@@ -81,7 +81,7 @@ if equal (TARGET, "MIPS64_OPENBSD") or e
         or equal (TARGET, "PA32_HPUX") or equal (TARGET, "PA64_HPUX")
         or equal (TARGET, "NT386") or equal (TARGET, "AMD64_FREEBSD")
         or equal (TARGET, "AMD64_OPENBSD") or equal (TARGET, "AMD64_NETBSD")
-        or equal (TARGET, "I386_INTERIX")
+        or equal (TARGET, "I386_INTERIX") or equal (TARGET, "AMD64_DRAGONFLY")
 
     Interface("Uucontext")
 
--- m3-libs/m3core/src/unix/m3makefile.orig	2009-07-28 18:49:13.000000000 +0000
+++ m3-libs/m3core/src/unix/m3makefile
@@ -6,6 +6,7 @@ readonly _UnixPieces = {
 % "AIX386"          : [ "aix-ps2-1-2",                             "uin-common" ],
 % "ALPHA_OSF"       : [ "osf-1.generic", "osf-1.ALPHA_OSF",        "uin-common" ],
   "AMD64_DARWIN"    : [ "darwin-common", "darwin-amd64",           "uin-len" ],
+  "AMD64_DRAGONFLY" : [ "freebsd-common",                          "uin-len" ],
   "AMD64_FREEBSD"   : [ "freebsd-common",                          "uin-len" ],
   "AMD64_LINUX"     : [ "linux-common",                            "uin-common" ],
   "AMD64_NETBSD"    : [ "netbsd-common",                           "uin-len" ],
--- m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY.orig	2014-01-16 23:14:23.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY
@@ -0,0 +1,9 @@
+readonly TARGET = "AMD64_DRAGONFLY"
+readonly GNU_PLATFORM = "amd64-dragonfly" % "cpu-os" string for GNU
+
+m3back_flags = "-gstabs+ -m64 -fPIC -funwind-tables"
+SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC"
+readonly SYSTEM_ASM = "as -64"
+
+include("AMD64.common")
+include("DragonFly.common")
--- m3-sys/cminstall/src/config-no-install/DragonFly.common.orig	2014-01-16 23:18:36.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/DragonFly.common
@@ -0,0 +1,24 @@
+readonly TARGET_OS = "DRAGONFLY"
+
+GNU_MAKE = "gmake"
+
+include("Unix.common")
+
+SYSTEM_LIBS{"ODBC"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lodbc" ]
+SYSTEM_LIBS{"POSTGRES95"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lpq" ]
+SYSTEM_LIBS{"X11"} = ["-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ]
+
+DRAGONFLY_CC_APPEND = " -z origin"
+                    & " -z now"
+                    & " -Bsymbolic"
+
+DRAGONFLY_LD_APPEND = DRAGONFLY_CC_APPEND
+                    & " -Wl,--fatal-warnings"
+                    & " -Wl,--warn-common"
+                    & " -Wl,-rpath,/usr/local/lib"
+                    & " -Wl,-rpath,\\$ORIGIN/../lib "
+
+SYSTEM_LD = SYSTEM_CC & DRAGONFLY_LD_APPEND
+SYSTEM_CC = SYSTEM_CC & DRAGONFLY_CC_APPEND
+
+include("gnuld.common")
--- m3-sys/m3cc/src/boot.py.orig	2008-05-19 13:09:03.000000000 +0000
+++ m3-sys/m3cc/src/boot.py
@@ -76,7 +76,7 @@ if SearchPath("gcc"):
     " + env)
 
   Run(gmake + "all-gmp all-mpfr all-libcpp all-libdecnumber \
-    all-build-libiberty all-libiberty configure-gcc")
+    all-build-libiberty all-libiberty configure-libcpp configure-gcc")
 
   os.chdir("gcc")
 
--- m3-sys/m3cc/src/m3makefile.orig	2010-05-14 07:00:52.000000000 +0000
+++ m3-sys/m3cc/src/m3makefile
@@ -577,7 +577,7 @@ end
 if GCC42
     m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
 else
-    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
+    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber configure-libcpp | tee -a " & Log])
 end
 
 %
--- m3-sys/m3cc/src/platforms.quake.orig	2009-07-29 17:11:08.000000000 +0000
+++ m3-sys/m3cc/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "i686-apple-darwin8",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd7",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3gdb/src/platforms.quake.orig	2010-04-21 17:06:38.000000000 +0000
+++ m3-sys/m3gdb/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "amd64-apple-darwin8.7.1",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3middle/src/Target.i3.orig	2010-02-01 14:05:35.000000000 +0000
+++ m3-sys/m3middle/src/Target.i3
@@ -33,7 +33,7 @@ TYPE
     AMD64_DARWIN, AMD64_LINUX, SPARC32_LINUX, SPARC64_LINUX,
     SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD, SPARC64_SOLARIS,
     I386_OPENBSD, AMD64_FREEBSD, PA32_HPUX, PA64_HPUX, ARM_DARWIN,
-    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, Undefined
+    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY, Undefined
   };
 
 CONST
@@ -89,7 +89,8 @@ CONST
     (* 48 *) "ARM_DARWIN",
     (* 49 *) "I386_INTERIX",
     (* 50 *) "AMD64_NETBSD",
-    (* 51 *) "AMD64_OPENBSD"
+    (* 51 *) "AMD64_OPENBSD",
+    (* 52 *) "AMD64_DRAGONFLY"
   };
 
 CONST
--- m3-sys/m3middle/src/Target.m3.orig	2010-02-01 15:06:05.000000000 +0000
+++ m3-sys/m3middle/src/Target.m3
@@ -307,6 +307,7 @@ PROCEDURE Init (system: TEXT; in_OS_name
 
     |  Systems.AMD64_NETBSD,
        Systems.AMD64_OPENBSD,
+       Systems.AMD64_DRAGONFLY,
        Systems.AMD64_FREEBSD =>
                  Jumpbuf_size              := 16_60 * Char.size;
 
--- m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c.orig	2009-04-12 10:31:23.000000000 +0000
+++ m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c
@@ -1,11 +1,11 @@
-#ifdef __FreeBSD__
+#if defined __FreeBSD__) || defined(__DragonFly__)
 #include <sys/types.h>
 #include <libutil.h>
 #endif
 
 void m3_setproctitle(const char *title)
 {
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
     setproctitle("%s", title);
 #endif
 }
--- scripts/boot-cm3-makefile-lib.tmpl.orig	2005-04-07 22:35:07.000000000 +0000
+++ scripts/boot-cm3-makefile-lib.tmpl
@@ -20,9 +20,10 @@ FILES_C  := $(wildcard *.c)
 FILES_S  := $(wildcard *.s)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C) $(FILES_S))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 
--- scripts/boot-cm3-makefile-prog.tmpl.orig	2003-07-10 22:16:05.000000000 +0000
+++ scripts/boot-cm3-makefile-prog.tmpl
@@ -19,9 +19,10 @@ FILES_MS := $(wildcard *.ms)
 FILES_C  := $(wildcard *.c)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 LIBFILES := $(shell for f in $(LIBS); do if [ -f "$${f}" ] ; then  echo $${f}; fi; done)
--- scripts/pack-crossbuild.sh.orig	2010-02-25 14:02:18.000000000 +0000
+++ scripts/pack-crossbuild.sh
@@ -23,7 +23,7 @@ if [ -z "$1" ] ; then
 fi
 CROSS_TARGET=$1
 BUILDARGS=''
-M3CONFIG=${ROOT}/m3-sys/cm3/src/config/${CROSS_TARGET}
+M3CONFIG=${ROOT}/m3-sys/cminstall/src/config-no-install/${CROSS_TARGET}
 export M3CONFIG
 . "$ROOT/scripts/pkginfo.sh"
 . "$ROOT/scripts/pkgcmds.sh"
@@ -38,25 +38,7 @@ P="${P} m3back"
 P="${P} m3front"
 P="${P} m3quake"
 P="${P} cm3"
-P="${P} m3scanner"
-P="${P} m3tools"
-P="${P} m3cgcat"
-P="${P} m3cggen"
-[ "${M3GDB}" = yes ] && P="${P} m3gdb"
 P="${P} m3bundle"
-P="${P} mklib"
-P="${P} fix_nl"
-P="${P} libdump"
-P="${P} bitvector"
-P="${P} digraph"
-P="${P} parseparams"
-P="${P} realgeometry"
-P="${P} set"
-P="${P} slisp"
-P="${P} sortedtableextras"
-P="${P} table-list"
-P="${P} tempfiles"
-[ "${HAVE_TCL}" = "yes" ] && P="${P} tcl"
 
 USAGE="
   `basename $0` cross_target

From jay.krell at cornell.edu  Sun Jan 19 12:35:54 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:35:54 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBA2EF.4020101@marino.st>
References: <52DBA2EF.4020101@marino.st>
Message-ID: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>

C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/013940cc/attachment-0001.html>

From jay.krell at cornell.edu  Sun Jan 19 12:40:17 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:40:17 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>

Your change looks about right.
 
Did you first "upgrade" and existing install, with these changes, and then cross?
That is what you must do.
 
 - Jay
 
From: jay.krell at cornell.edu
To: adacore at marino.st; m3devel at elegosoft.com
Subject: RE: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Date: Sun, 19 Jan 2014 11:35:54 +0000




C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/1e7c8efe/attachment-0001.html>

From adacore at marino.st  Sun Jan 19 12:53:53 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 12:53:53 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <52DBBCD1.7020904@marino.st>

On 1/19/2014 12:35, Jay K wrote:
> C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
> m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized
> target machin
> e: TARGET = ", s.target);
>  
>  
>  Please don't bother with the last release.
>  Please use the current CVS version.

But the last release is what is in FreeBSD ports, the basis of both
FreeBSD and DragonFly.  And ports doesn't build from repos normally,
although there is a facility to pull directly from github, but I don't
like that method myself.

If there was an official downloadable snapshot or release candidate, I
could use that.  Since 2010 was the last release, is another one coming
soon?


>  And please use the C backend.
>  Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?

Yes, that was one of the many patches in the attachment I send in the
previous mail.


>  Please use this as a guide for files to consider changing:
>  C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
> m3-comm\udp\test\src\m3makefile
> m3-libs\libm3\src\os\POSIX\m3makefile
> m3-libs\m3core\src\float\m3makefile
> m3-libs\m3core\src\thread\m3makefile
> m3-libs\m3core\src\thread\PTHREAD\m3makefile
> m3-libs\m3core\src\unix\m3makefile
> m3-sys\m3cc\src\m3makefile
> m3-sys\m3gdb\src\m3makefile
> m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
> m3-tools\cvsup\suplib\src\m3makefile
> m3-tools\cvsup\suptcp\src\POSIX\m3makefile

The only ones I haven't patched already are
m3-libs\m3core\src\thread\PTHREAD\m3makefile and the cvsup ones.  Those
wouldn't cause the error.  I also assume this is the top of the repo.


> C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
>   m3-libs\libm3\src\uid\POSIX\getMID.c 
>   m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c
>   m3-libs\m3core\src\Csupport\Common\s_lroundl.c
>  m3-libs\m3core\src\float\C99\FloatModeC.c 
>    m3-libs\m3core\src\runtime\common\RTProcessC.c 
>   m3-libs\m3core\src\runtime\POSIX\RTSignalC.c  
>   m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c
>   m3-libs\m3core\src\unix\Common\Uin.c
>   m3-libs\patternmatching\src\libglob\fnmatch.c
> 
>  
> esp. m3core and libm3.

I haven't seen freebsd in these, so must be a difference between release
and repo.



>  If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it
> for you.
>  But I realize there is much value in me NOT doing it.
>  I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok.

Because AMD64_DRAGONFLY is too long?
I'd prefer to keep it "AMD64_DRAGONFLY" because it matches the other
targets and because of all the work I've already done.


I'm really, really close to getting cm3 bootstrapped on DragonFly with
5.8.6.  I believe if this "M3_BOOTSTRAP = true" version would compile, I
could build cm3 and libraries on DragonFly with the cm3cg that's already
built on it.  Once I have a fully native compiler, then I can update to
the latest repo, especially if some kind of snapshot is released.

Thanks,
John


From jay.krell at cornell.edu  Sun Jan 19 13:00:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:00:01 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
Message-ID: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>

Windows uses SuspendThread.
I'll double check.


Cooperative suspend would NOT involve sending a "signal" to threads, in the Posix sense.
Nor would it use SuspendThread.


The model for cooperative suspend has been described by Tony and goes something like this:

There is a list of threads maintained by the m3 runtime.
  Not by enumerating OS threads.
The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.


When a thread wants to collect garbage, it either walks the thread list and sets all the booleans/enums to "suspend requested", or, more likely, it sets one global boolean.

Now, there are three likely methods to set the global boolean.
 1 - it is a global writable variable 
 2 - it is a global writable function pointer 
 3 - it is a page with protection 


Code frequently polls this global boolean, in one of the following fashions
 1 - read and test the value of the variable 
 2 - call the function pointer 
 3 - read the page (just a byte or a word) ignoring the contents 
 4 - call a function to do any of the above, esp #1 


#3 is probably fastest but my least favorite. It has lingering portability problems and is slowest when it comes time to actually suspend -- catch the exception and for purposes of debugging, you really don't want a design that raises exceptions like that.


When does the polling occur?
The polling is generated by the compiler.
It is done at function entry and/or exit and at branches or at least backward branches or at least branches that are continuing loops or such or at the starts of loops or somesuch.


I believe Java uses cooperative suspend and I think it uses the page protection mechanism.
It isn't portable e.g. to systems without an MMU.
Though of course you could abstract it out.


I'm glossing over details, but you know all the code we have for signals and semaphores? It goes away.
Our portability then would drive one more significant notch toward that of "hello world".

In my mine, we have about two pieces to do to gain "hello world" level of portability
  - cooperative suspend, not easy  
  - jmpbuf size removal, should be easy  

and some small matters
  - endianness, particularly in the libary 
  - word size 



 - Jay


From: dragisha at m3w.org
Date: Sat, 18 Jan 2014 09:24:11 +0100
To: m3devel at elegosoft.com
Subject: [M3devel] Cooperative suspend?

Jay,
What exactly do you mean? Isn't it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?
dd

--Dragi?a Duri?dragisha at m3w.org



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/88389edf/attachment-0001.html>

From adacore at marino.st  Sun Jan 19 13:09:37 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:09:37 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
Message-ID: <52DBC081.1070401@marino.st>

On 1/19/2014 12:40, Jay K wrote:
> Your change looks about right.
>  
> Did you first "upgrade" and existing install, with these changes, and
> then cross?
> That is what you must do.


It's probably easier that I show you.
I've attached "Makefile.local" (I added the .txt extension just now to
help out some mail clients) which is my recipe.

So I run three targets in a freshly patched workarea:
1) make cross-compiler
2) make assembler-code (called from cross-archive)
3) make cross-archive

In the cross-compiler target, the FreeBSD bootstrap compiler builds
m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.

The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
Everything is cleaned, then the cross compiler builds everything again
(including libm3core and libm3) along with m3objfile, sysutils, and
patternmatching.

Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
cm3cg is cm3cg-AMD64_DRAGONFLY.

I would not be surprised is there is a logic flaw or two in "make
cross-compiler".  The "how to port CM3" tutorial was slightly out of
date and some of the instructions were a bit ambiguous so this was a
best guess.

Regards,
John
-------------- next part --------------
# This makefile fragment is automatically included by bsd.port.mk
# It is separate because it contains maintenance targets not used in
# normal package building, but the recipes are too value to lose.  So
# we keep them with the port, but out of the way of regular users.
#
# $FreeBSD$


##
#
# The "create-bootstrap" target will assemble a new bootstrap based on the
# contents of the stage directory, which means it has to be run after the
# "stage" target.
#
##

NEWBOOTDIR=	${WRKDIR}/new-bootstrap/bootstrap
NEWBOOTNAME=	m3-bootstrap.${MARCH}.${OPSYS:U}.${OSREL:S/.//}.tar.bz2
BSCONTENTS=	bin/cm3 bin/cm3cg bin/m3bundle bin/mklib etc/modula3 \
		lib/libm3core.* lib/libm3.* lib/libsysutils.* \
		lib/libpatternmatching.* pkg/m3core pkg/libm3 pkg/sysutils \
		pkg/patternmatching pkg/m3middle pkg/m3objfile pkg/m3linker \
		pkg/m3back pkg/m3front pkg/m3quake pkg/cm3 pkg/mklib

create-bootstrap:
	@${RM} -rf ${NEWBOOTDIR}
	@${MKDIR} ${NEWBOOTDIR}/bin ${NEWBOOTDIR}/lib \
	          ${NEWBOOTDIR}/pkg ${NEWBOOTDIR}/etc
.for X in ${BSCONTENTS}
	@${CP} -a ${STAGEDIR}${PREFIX}/${X} ${NEWBOOTDIR}/${X:H}/
.endfor
	${ECHO} "INSTALL_ROOT = path() & \"/..\"" \
		>  ${NEWBOOTDIR}/bin/cm3.cfg
	${ECHO} "include(path() & \"/../etc/modula3/${M3TARGET}\")" \
		>> ${NEWBOOTDIR}/bin/cm3.cfg
	@${FIND} ${NEWBOOTDIR} -type f -perm +111 | ${XARGS} ${STRIP_CMD}
	(cd ${NEWBOOTDIR}/.. ; tar -cyf ${NEWBOOTNAME} bootstrap)


##
#
# The "cross-compiler" target will create a cross-compiler in the
# stage directory based on the value of ${CROSSTARG}.
# Execute it after the "patch" target
#
# The "cross-archive" will archive assembler code meant for linking
# on the native plaform.  It is run after "cross-compiler"
#
# The "native-link" target will bootstrap cm3 on a new platform (which
# obviously supports ports) using the archived files generated above.
# It could be done remotely via SSH, but this one target assumes files
# are locally available at ${WRKSRC}/${CROSSTARG}
#
##

CROSSTARG?=	AMD64_DRAGONFLY
CORELIBS=	m3-libs/m3core m3-libs/libm3
CORESYS=	m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker \
		m3-sys/m3back m3-sys/m3front m3-sys/m3quake m3-sys/cm3 \
		m3-tools/m3bundle
COREHELPERS=	m3-libs/sysutils m3-libs/patternmatching
COREFILES=	${CORELIBS} ${CORESYS}

cross-compiler:
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/build-cross-backend.sh ${CROSSTARG})
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/do-pkg.sh onlybuild \
		m3middle m3linker m3front m3quake cm3)
	@${FIND} ${WRKSRC} -name \.M3SHIP -print | ${XARGS} ${SED} -i -e \
		's|/bootstrap/|/stage${PREFIX}/|g'
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh ship \
		m3middle m3linker m3front m3quake cm3)	
	(cd ${WRKSRC}/m3-sys/cminstall/src/config-no-install && \
		${COPYTREE_SHARE} . ${STAGEDIR}${PREFIX}/etc/modula3)
	${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/cm3/${M3TARGET}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${M3TARGET}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${INSTALL_PROGRAM} ${WRKDIR}/bootstrap/bin/cm3cg \
		${STAGEDIR}${PREFIX}/bin/
	${INSTALL_PROGRAM} \
		${WRKSRC}/m3-sys/m3cc/${M3TARGET}-${CROSSTARG}/cm3cg \
		${STAGEDIR}${PREFIX}/bin/cm3cg-${CROSSTARG}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal \
		m3middle m3linker m3front m3quake cm3)
.for component in ${CORELIBS}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T})
.endfor
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${CORELIBS:T})
.for component in ${CORELIBS} ${COREHELPERS} ${CORESYS}
	@(cd ${WRKSRC}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${component:T} \
	)
.endfor
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "m3back = INSTALL_ROOT & \"/bin/cm3cg-${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "BUILD_DIR = \"${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "HOST = \"${M3TARGET}\"" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${CROSSTARG}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "M3_BOOTSTRAP = TRUE" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	@${ECHO} "====================================="
	@${ECHO} "=====  M3 cross-compiler ready  ====="
	@${ECHO} "====================================="
	@${ECHO}

assembler-code:
.for component in ${COREFILES}
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} SHIP=${TRUE} TARGET=${CROSSTARG} \
	    ${BUILTCM3} ${SH} scripts/do-pkg.sh onlybuild ${component:T} \
	)
.endfor
.for component in m3-libs/libm3 m3-sys/m3middle
	echo '#include "m3core.h"' > \
		${WRKSRC}/${component}/${CROSSTARG}/m3unix.h
.endfor
	${CP} ${WRKSRC}/m3-libs/m3core/src/m3core.h \
		${WRKSRC}/m3-libs/libm3/${CROSSTARG}/
	(cd ${WRKSRC}/m3-sys/m3middle; ${CP} src/POSIX/m3core.h ${CROSSTARG}/)

cross-archive: assembler-code
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} M3GDB=no ${BUILTCM3} \
		${SH} scripts/pack-crossbuild.sh ${CROSSTARG})

native-cm3cg:
# This target requires python to be installed
	@(cd ${WRKSRC}/m3-sys/m3cc; src/boot.py)
	@${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/m3cc/obj.1/gcc/m3cgc1 \
		${STAGEDIR}${PREFIX}/bin/cm3cg

native-link: native-cm3cg
.for component in ${COREFILES}
	@(cd ${WRKSRC}/${CROSSTARG}; ${TAR} -xzf ${component:T}.tgz)
.endfor
	@(cd ${WRKSRC}/scripts; \
		${SETENV} PATH=${PATH}:${STAGEDIR}${PREFIX}/bin \
		./boot-cm3-build-on-target.sh ${CROSSTARG})
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-sys/cm3/${CROSSTARG}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-tools/m3bundle/${CROSSTARG}/m3bundle \
		${STAGEDIR}${PREFIX}/bin

From jay.krell at cornell.edu  Sun Jan 19 13:17:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:17:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC081.1070401@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
Message-ID: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>

Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/0b2d21ba/attachment-0001.html>

From adacore at marino.st  Sun Jan 19 13:38:32 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:38:32 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <52DBC748.5090308@marino.st>

On 1/19/2014 13:17, Jay K wrote:
> Hey..I like that that does capture some of My Way. 
> Here is my independent off the cuff write up though:
>  I know the ports system is nice.
>  I know we really need to be in it for many operating systems.
>  I suspect your problem is that you didn't "upgrade" to apply you
> changes at the right time.

I must be misunderstanding what you mean by "upgrade".  It's the
cross-compiler that's complaining, and it was built by the latest patches.


>  Do any of these work for you:
>  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html

I'm not clear to which set you are referring.
There's no need for a binary set, modula3 now builds on FreeBSD 9 and
FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
superior to the release binaries because they link gmp and mpfr
statically while a bug in the cm3 sources caused those libraries to link
dynamically (which is why release binaries don't run on current FreeBSD)

Then there are the source distributions, but they are from 2012 so they
aren't exactly fresh either.


>  Anyway, please try this: 
>   Ignore DragonFly at first. 
>   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> anything. 
>   Such as from ports. 
>   Make a backup, e.g. of /usr/local/cm3. 
>   My instructions are unfortunately destructive of the existing install. 
>   Checkout the full current CVS repository. 
>   In that repository, run scripts/python/upgrade.py. 
>   If that doesn't work, stop, and we'll help. 

Is the goal just to get FreeBSD to have a binary with the latest
sources?  If so, I'd just modify the port to do that now that we have
working binary bootstraps.  The main reason I'm reluctant is that it
took over a week to make the cm3 port, I've never had so much difficulty
before.  (I've also spent another week on the DragonFly port).  So I'm
"resisting" because of how much work it already took.  That's why I was
trying to port DragonFly, THEN update to latest rather than update to
latest and then port DragonFly.  (in other words, going forward seems a
short path than backtracking)



>   If that does work, apply your diffs. And upgrade.py again. You can say
> "upgrade.py skipgcc" here. 
>   Then boot1.py c amd64_dragonflybsd   
>     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> on the command line. 
>     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> gcc/amd64 works for the same
>     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.

I assume you mean after the system understands this target.  There will
be a lot of files with __FreeBSD__ that need to be changed to
__FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
regenerate a bunch of patches from my current work.


>   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
>   Copy that to your Dragonfly system, extract it, cd into it, make. 
>   If that works, you have a native cm3. 
>   Run it. It should say "can't find cm3.cfg".  
>   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
>   cp cm3 /usr/local/cm3/bin  
>   export PATH=/usr/local/cm3/bin:$PATH
>   full CVS checkout  
>   cd scripts/python ; ./boot2.sh 
>    
>   You will need to have made small changes to pylib.py for this to work.
>   (And your config file should say to use the C backend, like Darwin and
> ARM_LINUX do.)


Well, I'm thrilled if I don't have to patch gcc again, that takes a long
time.  Shouldn't FreeBSD use the C backend too?

John


From jay.krell at cornell.edu  Sun Jan 19 13:32:44 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:32:44 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>, ,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>,
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>, ,
	<52DBC081.1070401@marino.st>,
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <COL130-W171FDF068BFCF6E20D357EE6BA0@phx.gbl>

Clarification:
upgrade.sh (note the .sh) rebuilds everything
I wrote upgrade.py based on it, but missed the "everything" part.
upgrade.py only upgrades "the compiler" (and m3core, libm3..)

A better practise although inefficient is:

 upgrade.py && ./do-cm3-all.py realclean skipgcc && ./do-cm3-all.py buildship  

I didn't suggest that to you here in order to get further in the pipeline faster.

However upgrade.py w/o other rebuild could leave m3core.so too new for many programs.
But for purposes of building/cross-building just the compiler, my instructions are good.


John, I see you added cm3 to the FreeBSD ports and you clearly know what you are doing.
Nevertheless, I am a lazy snob, don't quite see what you are doing wrong, and am suggesting "just different", not clearly better.
But, to my credit, this "just different" is what I use all the time.
I have used it to cross to several systems, w/ and w/o cm3cg (w/ and w/o C backend).
It isn't fast, and it doesn't always have the best incrementality, but it does usually work.
There is redundancy. pylib.py essentially duplicates parts of the config files.


You misunderstood what I meant by "C backend" and from the ports history you will somewhat appreciate it.
The C backend, instead of having a lot to do with gcc, is a backend that generates C, and then invokes a C compiler.
The problem you had with -gstabs for example, goes away. We'll use whatever debugging format regular C/C++ use.
There is a little hope of taking the one C output and compiling it for different targets -- at least FreeBSD 8,9,10.
Currently the C is very word size dependent, endian dependent, and somewhat otherwise target-dependent.
I'd like to fix these aspects but it isn't easy by far, will take possibly controversial changes in the frontend and interface to backend.


 - Jay




From: jay.krell at cornell.edu
To: adacore at marino.st
Date: Sun, 19 Jan 2014 12:17:24 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)




Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/153439e7/attachment-0001.html>

From jay.krell at cornell.edu  Sun Jan 19 13:50:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:50:27 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC748.5090308@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
Message-ID: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>

I understand your reluctance.


> Is the goal just to get FreeBSD to have a binary with the latest
> sources?


Yes, and to make sure the build process works for you.


> Shouldn't FreeBSD use the C backend too?


Every target should. I haven't convinced everyone yet.
There is less resistance for new targets, like AMD64_NT and ARM_LINUX, which are working, using the C backend.
There are downsides to the C backend:
  - it is slower to compiler  
  - debugging is degraded on systems that have m3gdb  
 (debugging is much better on systems w/o m3gdb) 


And there are advantages not yet implemented, in particular, efficient exception handling by generating C++.


> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.


Your patches should apply fairly cleanly to any new checkout.
I am willing to through them and apply them to CVS if you want -- depends on if you are ok with my stealing your credit, vs. you want to commit them yourselves.
I already skimmed them and they all look easy.
DragonflyBSD is basically FreeBSD by another name.
Yes, a ton of work has been done on the kernel, file systems, maybe ports.
But the ABI is almost the same -- heck, for the most part, Linux==NetBSD==FreeBSD==OpenBSD.
Sure, they might be implemented differently, but they are highly source compatible and every highly object code compatible.


> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.

Agreed, we are both missing something simple.
Usually this is the error you get when you don't upgrade Target.i3/Target.m3.
If you want to press on with the current approach and ignore my advise, it should be possible to step through all this.
To start, break on Target__Init and see if it returns true or false.


> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)


Ugh. I saw something about that in your diffs. Definitely we intend to link statically.
I actually went to the trouble of removing use of gmp/mpfr/mpc in current source.
They are used for actually very little and aren't worth it.


 - Jay


> Date: Sun, 19 Jan 2014 13:38:32 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 13:17, Jay K wrote:
> > Hey..I like that that does capture some of My Way. 
> > Here is my independent off the cuff write up though:
> >  I know the ports system is nice.
> >  I know we really need to be in it for many operating systems.
> >  I suspect your problem is that you didn't "upgrade" to apply you
> > changes at the right time.
> 
> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.
> 
> 
> >  Do any of these work for you:
> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
> 
> I'm not clear to which set you are referring.
> There's no need for a binary set, modula3 now builds on FreeBSD 9 and
> FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
> superior to the release binaries because they link gmp and mpfr
> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)
> 
> Then there are the source distributions, but they are from 2012 so they
> aren't exactly fresh either.
> 
> 
> >  Anyway, please try this: 
> >   Ignore DragonFly at first. 
> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> > anything. 
> >   Such as from ports. 
> >   Make a backup, e.g. of /usr/local/cm3. 
> >   My instructions are unfortunately destructive of the existing install. 
> >   Checkout the full current CVS repository. 
> >   In that repository, run scripts/python/upgrade.py. 
> >   If that doesn't work, stop, and we'll help. 
> 
> Is the goal just to get FreeBSD to have a binary with the latest
> sources?  If so, I'd just modify the port to do that now that we have
> working binary bootstraps.  The main reason I'm reluctant is that it
> took over a week to make the cm3 port, I've never had so much difficulty
> before.  (I've also spent another week on the DragonFly port).  So I'm
> "resisting" because of how much work it already took.  That's why I was
> trying to port DragonFly, THEN update to latest rather than update to
> latest and then port DragonFly.  (in other words, going forward seems a
> short path than backtracking)
> 
> 
> 
> >   If that does work, apply your diffs. And upgrade.py again. You can say
> > "upgrade.py skipgcc" here. 
> >   Then boot1.py c amd64_dragonflybsd   
> >     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> > on the command line. 
> >     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.
> 
> 
> >   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
> >   Copy that to your Dragonfly system, extract it, cd into it, make. 
> >   If that works, you have a native cm3. 
> >   Run it. It should say "can't find cm3.cfg".  
> >   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
> >   cp cm3 /usr/local/cm3/bin  
> >   export PATH=/usr/local/cm3/bin:$PATH
> >   full CVS checkout  
> >   cd scripts/python ; ./boot2.sh 
> >    
> >   You will need to have made small changes to pylib.py for this to work.
> >   (And your config file should say to use the C backend, like Darwin and
> > ARM_LINUX do.)
> 
> 
> Well, I'm thrilled if I don't have to patch gcc again, that takes a long
> time.  Shouldn't FreeBSD use the C backend too?
> 
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/ce74cb6a/attachment-0001.html>

From adacore at marino.st  Sun Jan 19 14:06:02 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 14:06:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <52DBCDBA.8030809@marino.st>

On 1/19/2014 13:50, Jay K wrote:
> And there are advantages not yet implemented, in particular, efficient
> exception handling by generating C++.

Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
dl_iterate_phdr which gcc uses for efficient zero-cost exception
handling.  That might come in handy here (I implemented it in
DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)


> Your patches should apply fairly cleanly to any new checkout.
> I am willing to through them and apply them to CVS if you want --
> depends on if you are ok with my stealing your credit, vs. you want to
> commit them yourselves.

I would love it if you did this - I'm happy with a mention in the commit
message.  I just attached a tarball of all the patches I'm using right
now.  Some of these changes are specifically for the cross compiler (I'm
think of the /scripts changes mostly) but most are valid assuming they
still apply to the head of the repo.  Please take as much as you want
with my gratitude.

> I already skimmed them and they all look easy.
> DragonflyBSD is basically FreeBSD by another name.

That's not an accident.  As a DragonFly developer that mostly works on
userland, I've been converging FreeBSD and DragonFly again.  They
drifted apart after 10 years, but I thought it better to keep the
userlands as similar as possible for 3rd party applications.

> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> But the ABI is almost the same -- heck, for the most part,
> Linux==NetBSD==FreeBSD==OpenBSD.
> Sure, they might be implemented differently, but they are highly source
> compatible and every highly object code compatible.
>> I must be misunderstanding what you mean by "upgrade". It's the
>> cross-compiler that's complaining, and it was built by the latest patches.
> Agreed, we are both missing something simple.
> Usually this is the error you get when you don't upgrade
> Target.i3/Target.m3.
> If you want to press on with the current approach and ignore my advise,

I'm not ignoring it; it's more like I'm not quite ready to give up since
I'm so close.  I think I could actually get these libs and cm3 to build
on DragonFly if I used AMD64_FREEBSD as the target with the
cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
I'd then get stuck on the DragonFly side trying to compile cm3 with
itself, theoretically I should get the same error there.


> Ugh. I saw something about that in your diffs. Definitely we intend to
> link statically.
> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> current source.
> They are used for actually very little and aren't worth it.

Even better!
Regards,
John

-------------- next part --------------
A non-text attachment was scrubbed...
Name: files.tar
Type: application/x-tar
Size: 13062 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/4ce4f917/attachment-0001.tar>

From dragisha at m3w.org  Sun Jan 19 15:28:18 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 19 Jan 2014 15:28:18 +0100
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
Message-ID: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>

Why would we do this, and not signalling, as in POSIX implementation?

Why try to make something synchronous when we have working asynchronous implementation?

Why would it be better than sync one?

On 19 Jan 2014, at 13:00, Jay K <jay.krell at cornell.edu> wrote:

> There is a list of threads maintained by the m3 runtime.
>   Not by enumerating OS threads.
> The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment-0001.sig>

From mika at async.caltech.edu  Sun Jan 19 21:05:48 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:05:48 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
Message-ID: <20140119200548.C18011A208F@async.async.caltech.edu>

"less worried" = "I think it works"?

I think the only threading that works in CM3 is user threading...

I tried to run CM3-compiled binaries through "valgrind" but got an
error that CM3 is using the same signal that valgrind does internally
(on AMD64_LINUX at least).  I think this is easy to change in m3core
(yes?) but haven't gotten around to it...

valgrind has a threading checker called "helgrind"...

     Mika

Jay writes:
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/plain;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>w up further.
>
> - Jay
>
>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>
>
>> Jay:
>> =20
>> I=E2=80=99m very concerned about the threading not working properly on bot=
>h 32-bit and 64-bit Windows.
>> =20
>> The Thread Test program crashes for me on both platforms.
>> =20
>> I haven=E2=80=99t tried your new test program yet.
>> =20
>> --Randy
>> =20
>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>> Sent: Saturday, January 18, 2014 1:23 AM
>> To: m3devel
>> Subject: EXT:[M3devel] Win32 SuspendThread?
>> =20
>>  This program also doesn't behave as expected, native, nothing to do with w=
>ow64.=20
>>  Anyone else please confirm:=20
>>    1) my expectations -- it should never print anything
>>    2) their importance  -- garbage collector depends on it =20
>>    3) their not being met -- stuff gets printed=20
>>  This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>  I am following up further.=20
>>  Maybe we should get cooperative suspend really going?=20
>> Thank you.
>>  - Jay=20
>> =20
>> #include <stdio.h>
>> #include <windows.h>
>> volatile long value;
>> unsigned long __stdcall Thread(PVOID parameter)
>> {
>>   while (1)
>>    InterlockedIncrement(&value);
>>   return 0;
>> }
>> int __cdecl main()
>> {
>>   HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>   UINT i =3D 0;
>>   while (1)
>>   {
>>     i +=3D 1;
>>     if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>     {
>>       printf("suspend failed %X\n", GetLastError());
>>       Sleep(1);
>>       continue;
>>     }
>>     volatile long a =3D value;=20
>>     volatile long b =3D value;
>>     if (a !=3D b)
>>     {
>>      printf("%d %d %d %d\n", i, a, b, b - a);
>>     }
>>     ResumeThread(thread);
>>   }
>> }
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/html;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
><blockquote type=3D"cite"><div>
>
><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>
><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
><style><!--
>/* Font Definitions */
>@font-face
>	{font-family:Calibri;
>	panose-1:2 15 5 2 2 2 4 3 2 4;}
>@font-face
>	{font-family:Tahoma;
>	panose-1:2 11 6 4 3 5 4 4 2 4;}
>/* Style Definitions */
>p.MsoNormal, li.MsoNormal, div.MsoNormal
>	{margin:0in;
>	margin-bottom:.0001pt;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>a:link, span.MsoHyperlink
>	{mso-style-priority:99;
>	color:blue;
>	text-decoration:underline;}
>a:visited, span.MsoHyperlinkFollowed
>	{mso-style-priority:99;
>	color:purple;
>	text-decoration:underline;}
>p
>	{mso-style-priority:99;
>	mso-margin-top-alt:auto;
>	margin-right:0in;
>	mso-margin-bottom-alt:auto;
>	margin-left:0in;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>	{mso-style-priority:99;
>	mso-style-link:"Balloon Text Char";
>	margin:0in;
>	margin-bottom:.0001pt;
>	font-size:8.0pt;
>	font-family:"Tahoma","sans-serif";}
>span.EmailStyle18
>	{mso-style-type:personal-reply;
>	font-family:"Calibri","sans-serif";
>	color:#1F497D;}
>span.BalloonTextChar
>	{mso-style-name:"Balloon Text Char";
>	mso-style-priority:99;
>	mso-style-link:"Balloon Text";
>	font-family:"Tahoma","sans-serif";}
>.MsoChpDefault
>	{mso-style-type:export-only;
>	font-size:10.0pt;}
>@page WordSection1
>	{size:8.5in 11.0in;
>	margin:1.0in 1.0in 1.0in 1.0in;}
>div.WordSection1
>	{page:WordSection1;}
>--></style><!--[if gte mso 9]><xml>
><o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
></xml><![endif]--><!--[if gte mso 9]><xml>
><o:shapelayout v:ext=3D"edit">
><o:idmap v:ext=3D"edit" data=3D"1" />
></o:shapelayout></xml><![endif]-->
>
>
><div class=3D"WordSection1">
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>shes for me on both platforms.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>r new test program yet.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><div>
><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>in 0in">
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
><b>On Behalf Of </b>Jay K<br>
><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
><b>To:</b> m3devel<br>
><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
></div>
></div>
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
><div>
><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>in-bottom:12.0pt;margin-left:.5in">
><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>;This program also doesn't behave as expected, native, nothing to do with wo=
>w64.
><br>
>&nbsp;Anyone else please confirm: <br>
>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>sp; <br>
>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>&nbsp;I am following up further. <br>
>&nbsp;Maybe we should get cooperative suspend really going? <br>
>Thank you.<br>
>&nbsp;- Jay <br>
>&nbsp;<br>
>#include &lt;stdio.h&gt;<br>
>#include &lt;windows.h&gt;<br>
>volatile long value;<br>
>unsigned long __stdcall Thread(PVOID parameter)<br>
>{<br>
>&nbsp; while (1)<br>
>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>&nbsp; return 0;<br>
>}<br>
>int __cdecl main()<br>
>{<br>
>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>&nbsp; UINT i =3D 0;<br>
>&nbsp; while (1)<br>
>&nbsp; {<br>
>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>;<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>&nbsp; }<br>
>}<o:p></o:p></span></p>
></div>
></div>
>
>
></div></blockquote></body></html>=
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From mika at async.caltech.edu  Sun Jan 19 21:13:08 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:13:08 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <20140119201308.0A4B41A208F@async.async.caltech.edu>


There were some bugs we never got around to addressing with the C backend,
weren't there?  A crash that didn't happen with native backend.

I've been using the "native" at more or less the head on very recent
Debian and the Raspberry Pi for a while now with no problems to speak of.
(As long as you only use user threads... but that's another story, I think.)

     Mika

Jay K writes:
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>I understand your reluctance.
>
>
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?
>
>
>Yes=2C and to make sure the build process works for you.
>
>
>> Shouldn't FreeBSD use the C backend too?
>
>
>Every target should. I haven't convinced everyone yet.
>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>which are working=2C using the C backend.
>There are downsides to the C backend:
>  - it is slower to compiler =20
>  - debugging is degraded on systems that have m3gdb =20
> (debugging is much better on systems w/o m3gdb)=20
>
>
>And there are advantages not yet implemented=2C in particular=2C efficient =
>exception handling by generating C++.
>
>
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>
>
>Your patches should apply fairly cleanly to any new checkout.
>I am willing to through them and apply them to CVS if you want -- depends o=
>n if you are ok with my stealing your credit=2C vs. you want to commit them=
> yourselves.
>I already skimmed them and they all look easy.
>DragonflyBSD is basically FreeBSD by another name.
>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>orts.
>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>Sure=2C they might be implemented differently=2C but they are highly source=
> compatible and every highly object code compatible.
>
>
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>
>Agreed=2C we are both missing something simple.
>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>3.
>If you want to press on with the current approach and ignore my advise=2C i=
>t should be possible to step through all this.
>To start=2C break on Target__Init and see if it returns true or false.
>
>
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>
>
>Ugh. I saw something about that in your diffs. Definitely we intend to link=
> statically.
>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>ource.
>They are used for actually very little and aren't worth it.
>
>
> - Jay
>
>
>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>> From: adacore at marino.st
>> To: jay.krell at cornell.edu
>> CC: m3devel at elegosoft.com
>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>=20
>> On 1/19/2014 13:17=2C Jay K wrote:
>> > Hey..I like that that does capture some of My Way.=20
>> > Here is my independent off the cuff write up though:
>> >  I know the ports system is nice.
>> >  I know we really need to be in it for many operating systems.
>> >  I suspect your problem is that you didn't "upgrade" to apply you
>> > changes at the right time.
>>=20
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>>=20
>>=20
>> >  Do any of these work for you:
>> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>=20
>> I'm not clear to which set you are referring.
>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>> superior to the release binaries because they link gmp and mpfr
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>>=20
>> Then there are the source distributions=2C but they are from 2012 so they
>> aren't exactly fresh either.
>>=20
>>=20
>> >  Anyway=2C please try this:=20
>> >   Ignore DragonFly at first.=20
>> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>> > anything.=20
>> >   Such as from ports.=20
>> >   Make a backup=2C e.g. of /usr/local/cm3.=20
>> >   My instructions are unfortunately destructive of the existing install=
>.=20
>> >   Checkout the full current CVS repository.=20
>> >   In that repository=2C run scripts/python/upgrade.py.=20
>> >   If that doesn't work=2C stop=2C and we'll help.=20
>>=20
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?  If so=2C I'd just modify the port to do that now that we have
>> working binary bootstraps.  The main reason I'm reluctant is that it
>> took over a week to make the cm3 port=2C I've never had so much difficult=
>y
>> before.  (I've also spent another week on the DragonFly port).  So I'm
>> "resisting" because of how much work it already took.  That's why I was
>> trying to port DragonFly=2C THEN update to latest rather than update to
>> latest and then port DragonFly.  (in other words=2C going forward seems a
>> short path than backtracking)
>>=20
>>=20
>>=20
>> >   If that does work=2C apply your diffs. And upgrade.py again. You can =
>say
>> > "upgrade.py skipgcc" here.=20
>> >   Then boot1.py c amd64_dragonflybsd  =20
>> >     Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>re
>> > on the command line.=20
>> >     Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>=2C
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>>=20
>>=20
>> >   If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>gz=20
>> >   Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>e.=20
>> >   If that works=2C you have a native cm3.=20
>> >   Run it. It should say "can't find cm3.cfg". =20
>> >   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>> >   cp cm3 /usr/local/cm3/bin =20
>> >   export PATH=3D/usr/local/cm3/bin:$PATH
>> >   full CVS checkout =20
>> >   cd scripts/python =3B ./boot2.sh=20
>> >   =20
>> >   You will need to have made small changes to pylib.py for this to work=
>.
>> >   (And your config file should say to use the C backend=2C like Darwin =
>and
>> > ARM_LINUX do.)
>>=20
>>=20
>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>ong
>> time.  Shouldn't FreeBSD use the C backend too?
>>=20
>> John
> 		 	   		  =
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 12pt=3B
>font-family:Calibri
>}
>--></style></head>
><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>h are working=2C using the C backend.<br>There are downsides to the C backe=
>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>et implemented=2C in particular=2C efficient exception handling by generati=
>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>=3B     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>=3B <br>&gt=3B I assume you mean after the system understands this target. =
> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'=
>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
> am willing to through them and apply them to CVS if you want -- depends on=
> if you are ok with my stealing your credit=2C vs. you want to commit them =
>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>e highly source compatible and every highly object code compatible.<br><br>=
><br>&gt=3B I must be misunderstanding what you mean by "upgrade".  It's the=
><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>r>If you want to press on with the current approach and ignore my advise=2C=
> it should be possible to step through all this.<br>To start=2C break on Ta=
>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
> to link statically.<br>I actually went to the trouble of removing use of g=
>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>dependent off the cuff write up though:<br>&gt=3B &gt=3B  I know the ports =
>system is nice.<br>&gt=3B &gt=3B  I know we really need to be in it for man=
>y operating systems.<br>&gt=3B &gt=3B  I suspect your problem is that you d=
>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade".  =
>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Do any of thes=
>e work for you:<br>&gt=3B &gt=3B  https://modula3.elegosoft.com/cm3/snaps/s=
>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Anyway=2C plea=
>se try this: <br>&gt=3B &gt=3B   Ignore DragonFly at first. <br>&gt=3B &gt=
>=3B   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B   Such as from ports. <br>&gt=3B =
>&gt=3B   Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B   My in=
>structions are unfortunately destructive of the existing install. <br>&gt=
>=3B &gt=3B   Checkout the full current CVS repository. <br>&gt=3B &gt=3B   =
>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B   If=
> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>?  If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>working binary bootstraps.  The main reason I'm reluctant is that it<br>&gt=
>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>lty<br>&gt=3B before.  (I've also spent another week on the DragonFly port)=
>.  So I'm<br>&gt=3B "resisting" because of how much work it already took.  =
>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>t rather than update to<br>&gt=3B latest and then port DragonFly.  (in othe=
>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B   If that does work=2C appl=
>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>y skipgcc" here. <br>&gt=3B &gt=3B   Then boot1.py c amd64_dragonflybsd   <=
>br>&gt=3B &gt=3B     Make sure you say "c" and the target "amd64_dragonflyb=
>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B    =
> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B     for Linux=2C=
> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>me you mean after the system understands this target.  There will<br>&gt=3B=
> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to<br>&gt=3B =
>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>r>&gt=3B &gt=3B   If this works=2C it will give you=2C roughly amd64_dragon=
>flybsd*.tar.gz <br>&gt=3B &gt=3B   Copy that to your Dragonfly system=2C ex=
>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B   If that works=2C you ha=
>ve a native cm3. <br>&gt=3B &gt=3B   Run it. It should say "can't find cm3.=
>cfg".  <br>&gt=3B &gt=3B   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
><br>&gt=3B &gt=3B   cp cm3 /usr/local/cm3/bin  <br>&gt=3B &gt=3B   export P=
>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B   full CVS checkout  <br>&g=
>t=3B &gt=3B   cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B    <br>&gt=
>=3B &gt=3B   You will need to have made small changes to pylib.py for this =
>to work.<br>&gt=3B &gt=3B   (And your config file should say to use the C b=
>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>hat takes a long<br>&gt=3B time.  Shouldn't FreeBSD use the C backend too?<=
>br>&gt=3B <br>&gt=3B John<br></div> 		 	   		  </div></body>
></html>=
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_--


From mika at async.caltech.edu  Sun Jan 19 22:36:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 13:36:49 -0800
Subject: [M3devel] narrow still failing..
Message-ID: <20140119213649.403AE1A208F@async.async.caltech.edu>


A smaller test:

     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  PROCEDURE Z(VAR x : REFANY) =
    20    BEGIN
    21      x := BDD.Format(x)
    22    END Z;
    23
    24  BEGIN
    25    Q()
    26  END Main.

(191)rover:~/refany/src>../FreeBSD4/refany

***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 21
***

What's going on here???

Here are the relevant declarations:

(* simple BDD package *)

INTERFACE BDD;
IMPORT Word;

TYPE 
  T <: REFANY;

(* the boolean constants top and bottom *)
PROCEDURE True() : T;
PROCEDURE False() : T;

(* a new variable *)
PROCEDURE New(name : TEXT := NIL) : T;

(* unary ops *)
PROCEDURE Not(a : T) : T;

(* binary ops *)
PROCEDURE And(a, b : T) : T;
PROCEDURE Xor(a , b : T) : T;
PROCEDURE Equivalent(a, b : T) : T;
PROCEDURE Or( a, b : T) : T;
PROCEDURE Implies  (a , b : T) : T;

(* maketrue/makefalse *)

PROCEDURE MakeFalse(b, v : T) : T; (* make v false in b *)
PROCEDURE MakeTrue(b, v : T) : T; (* make v true in b *)


(* print with ids *)
PROCEDURE Format(a : T; symtab : REFANY (* BDDTextTbl.T *) := NIL) : TEXT;

(* the following procedures allow this interface to be used in generics *)
PROCEDURE Equal(a, b : T) : BOOLEAN;
PROCEDURE Hash(a : T) : Word.T;

PROCEDURE Size(a : T) : CARDINAL;
  (* number of nodes in structure *)

CONST
  Brand = "BDD 0.1";

PROCEDURE GetId(a : T) : INTEGER; (* for debugging *)

END BDD.
....

REVEAL
  T = BRANDED Brand OBJECT
    l , r : T;
    root : Root;
    tag : CARDINAL; (* for hashing *)
    name : TEXT;
  METHODS
    init() : T := Init;
  END;

PROCEDURE New(name : TEXT) : T =
  VAR res : Root := NEW(Root).init(); BEGIN
    res.name := name;
    res.root := res;
    res.l := true; res.r := false;
    res.id := nextId;
    res.tab := NEW(BDDTripleHash.Default).init(128);
    FOR i := FIRST(res.cache) TO LAST(res.cache) DO
      res.cache[i] := NEW(BDDTripleHash.Default).init(64)
    END; 
    EVAL BDDTripleHash.Put(res.tab, Pair { res.l, res.r }, (res));
    INC(nextId);
    RETURN res
  END New;

PROCEDURE Format(x : T; symtab : REFANY := NIL) : TEXT =
  VAR nm : TEXT;
  BEGIN
    IF symtab # NIL AND NARROW(symtab, BDDTextTbl.T).get(x, nm) THEN
      RETURN nm
    END;
    IF x = true THEN RETURN "TRUE"
    ELSIF x = false THEN RETURN "FALSE"
    END;
       
    IF x.name # NIL THEN RETURN x.name END;

    RETURN Fmt.Int(x.root.id) & " && (" & Format(x.l) & ") || (" & Format(x.r) &
           ") && ~" & Fmt.Int(x.root.id)
  END Format;

  use option @M3stackdump to get a stack trace
Abort


I'm happy to share the whole package if anyone with a better clue than me would like a look.

In fact:

http://rover.gcapltd.com/~mika/bdd.tgz

It should build without anything outside of standard cm3. 

     Mika


From mika at async.caltech.edu  Sun Jan 19 23:01:23 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 14:01:23 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119213649.403AE1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
Message-ID: <20140119220123.0D1971A208F@async.async.caltech.edu>

As in the ISTYPE/TYPECASE example, the following works:


PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(b)
    END;
  END Z;

(244)rover:~/refany/src>../FreeBSD4/refany
a


This does NOT work:

PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(x)
    END;
  END Z;


(242)rover:~/refany/src>../FreeBSD4/refany


***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 22
***

  use option @M3stackdump to get a stack trace
Abort


Same result PM3 / CM3.

     Mika


From jay.krell at cornell.edu  Sun Jan 19 23:58:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 22:58:27 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>,
	<DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
Message-ID: <COL130-W23436BD09BE1F20C442D6FE6BA0@phx.gbl>

1) portability; We currently have separate code, roughly, for MacOSX, FreeBSD, OpenBSD, NT, and the other Posixes.
MacOSX doesn't have Posix semaphores, for example.
This would give us one codebase here.
Only IA64 would likely remain an outlier, because stack is not merely the address of one local to another, but includes also the grow-up stack for the register saves.



2) It'd workaround the wow64 problem.


?- Jay

________________________________
> Subject: Re: [M3devel] Cooperative suspend? 
> From: dragisha at m3w.org 
> Date: Sun, 19 Jan 2014 15:28:18 +0100 
> CC: m3devel at elegosoft.com 
> To: jay.krell at cornell.edu 
> 
> Why would we do this, and not signalling, as in POSIX implementation? 
> 
> Why try to make something synchronous when we have working asynchronous 
> implementation? 
> 
> Why would it be better than sync one? 
> 
> On 19 Jan 2014, at 13:00, Jay K 
> <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote: 
> 
> There is a list of threads maintained by the m3 runtime. 
> Not by enumerating OS threads. 
> The thread structure contains in it perhaps a boolean or enum as to 
> suspend requested, suspending, suspended, resuming, running, etc. 
> 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:13:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:13:24 +0000
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>,
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <COL130-W156EFA4699196726508DB4E6BA0@phx.gbl>

"not worried" for native windows.
Still worried about wow64.


The answer for native is in Rotor/sscli and I have a program to show it working/not-working.
Specifically SuspendThread needs to be followed by GetThreadContext to ensure the suspend has completed.
If we don't do that, we must. If we already do, we are ok. I'll look later.
wow64 is still a problem. I'm still looking into it and thinking about it. But we do have native AMD64_NT
so just saying wow64 doesn't work might be the answer. I know people will protest that it works, but doesn't always.
I haven't figured out what rotor/sscli does.
Java uses cooperative suspend, which sidesteps this stuff.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Win32 SuspendThread?
> Date: Sun, 19 Jan 2014 12:05:48 -0800
> From: mika at async.caltech.edu
>
> "less worried" = "I think it works"?
>
> I think the only threading that works in CM3 is user threading...
>
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least). I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
>
> valgrind has a threading checker called "helgrind"...
>
> Mika
>
> Jay writes:
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/plain;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>>w up further.
>>
>> - Jay
>>
>>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>>
>>
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>>h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>>ow64.=20
>>> Anyone else please confirm:=20
>>> 1) my expectations -- it should never print anything
>>> 2) their importance -- garbage collector depends on it =20
>>> 3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>> while (1)
>>> InterlockedIncrement(&value);
>>> return 0;
>>> }
>>> int __cdecl main()
>>> {
>>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>> UINT i =3D 0;
>>> while (1)
>>> {
>>> i +=3D 1;
>>> if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>> {
>>> printf("suspend failed %X\n", GetLastError());
>>> Sleep(1);
>>> continue;
>>> }
>>> volatile long a =3D value;=20
>>> volatile long b =3D value;
>>> if (a !=3D b)
>>> {
>>> printf("%d %d %d %d\n", i, a, b, b - a);
>>> }
>>> ResumeThread(thread);
>>> }
>>> }
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/html;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>><blockquote type=3D"cite"><div>
>>
>><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>>
>><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>><style>
>>
>>
>><div class=3D"WordSection1">
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>>:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>>shes for me on both platforms.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>>r new test program yet.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>>p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><div>
>><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>>in 0in">
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>><b>On Behalf Of </b>Jay K<br>
>><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>><b>To:</b> m3devel<br>
>><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>></div>
>></div>
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>><div>
>><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>>in-bottom:12.0pt;margin-left:.5in">
>><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>>;This program also doesn't behave as expected, native, nothing to do with wo=
>>w64.
>><br>
>>&nbsp;Anyone else please confirm: <br>
>>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>>sp; <br>
>>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>>&nbsp;I am following up further. <br>
>>&nbsp;Maybe we should get cooperative suspend really going? <br>
>>Thank you.<br>
>>&nbsp;- Jay <br>
>>&nbsp;<br>
>>#include &lt;stdio.h&gt;<br>
>>#include &lt;windows.h&gt;<br>
>>volatile long value;<br>
>>unsigned long __stdcall Thread(PVOID parameter)<br>
>>{<br>
>>&nbsp; while (1)<br>
>>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>>&nbsp; return 0;<br>
>>}<br>
>>int __cdecl main()<br>
>>{<br>
>>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>>&nbsp; UINT i =3D 0;<br>
>>&nbsp; while (1)<br>
>>&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>>;<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>>&nbsp; }<br>
>>}<o:p></o:p></span></p>
>></div>
>></div>
>>
>>
>></div></blockquote></body></html>=
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:15:02 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:15:02 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <20140119201308.0A4B41A208F@async.async.caltech.edu>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
Message-ID: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>

The only problem I remember is an alignment matter using msvcrt.dll on AMD64_NT.
I'll look into it, and the old emails.
I've used the C backend on many platforms now.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com; adacore at marino.st
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> Date: Sun, 19 Jan 2014 12:13:08 -0800
> From: mika at async.caltech.edu
>
>
> There were some bugs we never got around to addressing with the C backend,
> weren't there? A crash that didn't happen with native backend.
>
> I've been using the "native" at more or less the head on very recent
> Debian and the Raspberry Pi for a while now with no problems to speak of.
> (As long as you only use user threads... but that's another story, I think.)
>
> Mika
>
> Jay K writes:
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/plain; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>>I understand your reluctance.
>>
>>
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources?
>>
>>
>>Yes=2C and to make sure the build process works for you.
>>
>>
>>> Shouldn't FreeBSD use the C backend too?
>>
>>
>>Every target should. I haven't convinced everyone yet.
>>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>>which are working=2C using the C backend.
>>There are downsides to the C backend:
>> - it is slower to compiler =20
>> - debugging is degraded on systems that have m3gdb =20
>> (debugging is much better on systems w/o m3gdb)=20
>>
>>
>>And there are advantages not yet implemented=2C in particular=2C efficient =
>>exception handling by generating C++.
>>
>>
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>
>>
>>Your patches should apply fairly cleanly to any new checkout.
>>I am willing to through them and apply them to CVS if you want -- depends o=
>>n if you are ok with my stealing your credit=2C vs. you want to commit them=
>> yourselves.
>>I already skimmed them and they all look easy.
>>DragonflyBSD is basically FreeBSD by another name.
>>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>>orts.
>>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>>Sure=2C they might be implemented differently=2C but they are highly source=
>> compatible and every highly object code compatible.
>>
>>
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>
>>Agreed=2C we are both missing something simple.
>>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>>3.
>>If you want to press on with the current approach and ignore my advise=2C i=
>>t should be possible to step through all this.
>>To start=2C break on Target__Init and see if it returns true or false.
>>
>>
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>
>>
>>Ugh. I saw something about that in your diffs. Definitely we intend to link=
>> statically.
>>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>>ource.
>>They are used for actually very little and aren't worth it.
>>
>>
>> - Jay
>>
>>
>>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>>> From: adacore at marino.st
>>> To: jay.krell at cornell.edu
>>> CC: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>>=20
>>> On 1/19/2014 13:17=2C Jay K wrote:
>>>> Hey..I like that that does capture some of My Way.=20
>>>> Here is my independent off the cuff write up though:
>>>> I know the ports system is nice.
>>>> I know we really need to be in it for many operating systems.
>>>> I suspect your problem is that you didn't "upgrade" to apply you
>>>> changes at the right time.
>>>=20
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>>=20
>>>=20
>>>> Do any of these work for you:
>>>> https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>>=20
>>> I'm not clear to which set you are referring.
>>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>>> superior to the release binaries because they link gmp and mpfr
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>>=20
>>> Then there are the source distributions=2C but they are from 2012 so they
>>> aren't exactly fresh either.
>>>=20
>>>=20
>>>> Anyway=2C please try this:=20
>>>> Ignore DragonFly at first.=20
>>>> Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>>>> anything.=20
>>>> Such as from ports.=20
>>>> Make a backup=2C e.g. of /usr/local/cm3.=20
>>>> My instructions are unfortunately destructive of the existing install=
>>.=20
>>>> Checkout the full current CVS repository.=20
>>>> In that repository=2C run scripts/python/upgrade.py.=20
>>>> If that doesn't work=2C stop=2C and we'll help.=20
>>>=20
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources? If so=2C I'd just modify the port to do that now that we have
>>> working binary bootstraps. The main reason I'm reluctant is that it
>>> took over a week to make the cm3 port=2C I've never had so much difficult=
>>y
>>> before. (I've also spent another week on the DragonFly port). So I'm
>>> "resisting" because of how much work it already took. That's why I was
>>> trying to port DragonFly=2C THEN update to latest rather than update to
>>> latest and then port DragonFly. (in other words=2C going forward seems a
>>> short path than backtracking)
>>>=20
>>>=20
>>>=20
>>>> If that does work=2C apply your diffs. And upgrade.py again. You can =
>>say
>>>> "upgrade.py skipgcc" here.=20
>>>> Then boot1.py c amd64_dragonflybsd =20
>>>> Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>>re
>>>> on the command line.=20
>>>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>>=2C
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>>=20
>>>=20
>>>> If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>>gz=20
>>>> Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>>e.=20
>>>> If that works=2C you have a native cm3.=20
>>>> Run it. It should say "can't find cm3.cfg". =20
>>>> Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>>>> cp cm3 /usr/local/cm3/bin =20
>>>> export PATH=3D/usr/local/cm3/bin:$PATH
>>>> full CVS checkout =20
>>>> cd scripts/python =3B ./boot2.sh=20
>>>> =20
>>>> You will need to have made small changes to pylib.py for this to work=
>>.
>>>> (And your config file should say to use the C backend=2C like Darwin =
>>and
>>>> ARM_LINUX do.)
>>>=20
>>>=20
>>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>>ong
>>> time. Shouldn't FreeBSD use the C backend too?
>>>=20
>>> John
>> =
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/html; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>><html>
>><head>
>><style></style></head>
>><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>>h are working=2C using the C backend.<br>There are downsides to the C backe=
>>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
>> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>>et implemented=2C in particular=2C efficient exception handling by generati=
>>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>>=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>>=3B <br>&gt=3B I assume you mean after the system understands this target. =
>> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'=
>>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
>> am willing to through them and apply them to CVS if you want -- depends on=
>> if you are ok with my stealing your credit=2C vs. you want to commit them =
>>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>>e highly source compatible and every highly object code compatible.<br><br>=
>><br>&gt=3B I must be misunderstanding what you mean by "upgrade". It's the=
>><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>>r>If you want to press on with the current approach and ignore my advise=2C=
>> it should be possible to step through all this.<br>To start=2C break on Ta=
>>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
>><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
>> to link statically.<br>I actually went to the trouble of removing use of g=
>>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
>> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>>dependent off the cuff write up though:<br>&gt=3B &gt=3B I know the ports =
>>system is nice.<br>&gt=3B &gt=3B I know we really need to be in it for man=
>>y operating systems.<br>&gt=3B &gt=3B I suspect your problem is that you d=
>>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade". =
>>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
>> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Do any of thes=
>>e work for you:<br>&gt=3B &gt=3B https://modula3.elegosoft.com/cm3/snaps/s=
>>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
>> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Anyway=2C plea=
>>se try this: <br>&gt=3B &gt=3B Ignore DragonFly at first. <br>&gt=3B &gt=
>>=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B Such as from ports. <br>&gt=3B =
>>&gt=3B Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B My in=
>>structions are unfortunately destructive of the existing install. <br>&gt=
>>=3B &gt=3B Checkout the full current CVS repository. <br>&gt=3B &gt=3B =
>>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B If=
>> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>>? If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>>working binary bootstraps. The main reason I'm reluctant is that it<br>&gt=
>>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>>lty<br>&gt=3B before. (I've also spent another week on the DragonFly port)=
>>. So I'm<br>&gt=3B "resisting" because of how much work it already took. =
>>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>>t rather than update to<br>&gt=3B latest and then port DragonFly. (in othe=
>>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B If that does work=2C appl=
>>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>>y skipgcc" here. <br>&gt=3B &gt=3B Then boot1.py c amd64_dragonflybsd <=
>>br>&gt=3B &gt=3B Make sure you say "c" and the target "amd64_dragonflyb=
>>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B =
>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B for Linux=2C=
>> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>>me you mean after the system understands this target. There will<br>&gt=3B=
>> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>>FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to<br>&gt=3B =
>>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>>r>&gt=3B &gt=3B If this works=2C it will give you=2C roughly amd64_dragon=
>>flybsd*.tar.gz <br>&gt=3B &gt=3B Copy that to your Dragonfly system=2C ex=
>>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B If that works=2C you ha=
>>ve a native cm3. <br>&gt=3B &gt=3B Run it. It should say "can't find cm3.=
>>cfg". <br>&gt=3B &gt=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
>><br>&gt=3B &gt=3B cp cm3 /usr/local/cm3/bin <br>&gt=3B &gt=3B export P=
>>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B full CVS checkout <br>&g=
>>t=3B &gt=3B cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B <br>&gt=
>>=3B &gt=3B You will need to have made small changes to pylib.py for this =
>>to work.<br>&gt=3B &gt=3B (And your config file should say to use the C b=
>>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>>hat takes a long<br>&gt=3B time. Shouldn't FreeBSD use the C backend too?<=
>>br>&gt=3B <br>&gt=3B John<br></div> </div></body>
>></html>=
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- 		 	   		  

From mika at async.caltech.edu  Mon Jan 20 00:39:38 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 15:39:38 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
	<COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
Message-ID: <20140119233938.9C5A31A208F@async.async.caltech.edu>

Jay K writes:
>The only problem I remember is an alignment matter using msvcrt.dll on AMD6=
>4_NT.=0A=
>I'll look into it=2C and the old emails.=0A=
>I've used the C backend on many platforms now.=0A=

This stuff doesn't happen with the native backend:

> 2. I can't get anything with Trestle working. I'm running in a VNC
> session on a remote machine. Not sure this has anything to do with the
> compiler or even the ARM. Has anyone seen it before (on any system)?
>
> (gdb) cont
> Continuing.
> [New Thread 0x42276430 (LWP 24451)]
> [Thread 0x42276430 (LWP 24451) exited]
> [New Thread 0x424c6430 (LWP 24454)]
>
>
> ***
> *** runtime error:
> *** Exception "ZChildVBT.BadPercentage" not in RAISES list
> *** file "../src/lego/ZChildVBT.m3", line 61
> ***
>
>
> Program received signal SIGABRT, Aborted.
>
> backtrace:
>
> (gdb) where
> #0 ZChildVBT__Init (z_L_120=0x422bbf70 "H\357K", ch_L_121=0x422bdb84 "\260\326K", h_L_122=0, v_L_123=1.75, loc_L_124=4 '\004', type_L_125=1 '\001', shaper_
L_126=0x41a32b54, open_L_127=0 '\000') at ../src/lego/ZCh
> ildVBT.m3:61
> #1 0x40637518 in FormsVBT__pZChild (cl_L_1042=0x42280444, list_L_1043=0x424c54c4, s_L_1044=0x424c55dc) at ../src/FormsVBT.m3:2593
> #2 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a45994 "", state_L_303=0x424c55dc) at ../src/FormsVBT.m3:250
> #3 0x40648b3c in FormsVBT__AddChildren (cl_L_1229=0x42280444, v_L_1230=0x422b0634 " \365K", list_L_1231=0x41a48504, state_L_1232=0x424c55dc) at ../src/FormsVBT.m3:3671
> #4 0x40634138 in FormsVBT__pZSplit (cl_L_999=0x42280444, list_L_1000=0x424c56bc, s_L_1001=0x424c57c4) at ../src/FormsVBT.m3:2454
> #5 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a40ec0 "", state_L_303=0x424c57c4) at ../src/FormsVBT.m3:250
> #6 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c57c4) at ../src/FormsVBT.m3:3642
> #7 0x406129c0 in FormsVBT__pShape (cl_L_496=0x42280444, list_L_497=0x424c58a4, s_L_498=0x424c59a4) at ../src/FormsVBT.m3:948
> #8 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a402a4 "", state_L_303=0x424c59a4) at ../src/FormsVBT.m3:250
> #9 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c59a4) at ../src/FormsVBT.m3:3642
> #10 0x4062ca10 in FormsVBT__pScale (cl_L_905=0x42280444, list_L_906=0x424c5a84, s_L_907=0x42280458) at ../src/FormsVBT.m3:2165
> #11 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a4006c "", state_L_303=0x42280458) at ../src/FormsVBT.m3:250
> #12 0x40606dec in FormsVBT__Apply (cl_L_293=0x42280444) at ../src/FormsVBT.m3:84
> #13 0x40d27bd4 in ThreadPThread__RunThread (me_L_231=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:449
> #14 0x40d27590 in ThreadPThread__ThreadBase (param_L_229=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:422
> #15 0x41850bfc in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
> #16 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> #17 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> (gdb) cont
> Continuing.
>



From jay.krell at cornell.edu  Mon Jan 20 01:06:27 2014
From: jay.krell at cornell.edu (Jay)
Date: Sun, 19 Jan 2014 16:06:27 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <FC1A38F9-1A18-4DD1-99BB-9FB53669C761@gmail.com>

Cooperative suspend will also fix this signal allocation vs. valgrind problem. Can also likely pick a different one...

 - Jay

On Jan 19, 2014, at 12:05 PM, <mika at async.caltech.edu> wrote:

> "less worried" = "I think it works"?
> 
> I think the only threading that works in CM3 is user threading...
> 
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least).  I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
> 
> valgrind has a threading checker called "helgrind"...
> 
>     Mika
> 
> Jay writes:
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/plain;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>> w up further.
>> 
>> - Jay
>> 
>> On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>> 
>> 
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>> h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>> ow64.=20
>>> Anyone else please confirm:=20
>>>   1) my expectations -- it should never print anything
>>>   2) their importance  -- garbage collector depends on it =20
>>>   3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>>  while (1)
>>>   InterlockedIncrement(&value);
>>>  return 0;
>>> }
>>> int __cdecl main()
>>> {
>>>  HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>>  UINT i =3D 0;
>>>  while (1)
>>>  {
>>>    i +=3D 1;
>>>    if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>>    {
>>>      printf("suspend failed %X\n", GetLastError());
>>>      Sleep(1);
>>>      continue;
>>>    }
>>>    volatile long a =3D value;=20
>>>    volatile long b =3D value;
>>>    if (a !=3D b)
>>>    {
>>>     printf("%d %d %d %d\n", i, a, b, b - a);
>>>    }
>>>    ResumeThread(thread);
>>>  }
>>> }
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/html;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>> utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>> ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>> div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>> mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>> <blockquote type=3D"cite"><div>
>> 
>> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>> 
>> <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>> <style><!--
>> /* Font Definitions */
>> @font-face
>>    {font-family:Calibri;
>>    panose-1:2 15 5 2 2 2 4 3 2 4;}
>> @font-face
>>    {font-family:Tahoma;
>>    panose-1:2 11 6 4 3 5 4 4 2 4;}
>> /* Style Definitions */
>> p.MsoNormal, li.MsoNormal, div.MsoNormal
>>    {margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> a:link, span.MsoHyperlink
>>    {mso-style-priority:99;
>>    color:blue;
>>    text-decoration:underline;}
>> a:visited, span.MsoHyperlinkFollowed
>>    {mso-style-priority:99;
>>    color:purple;
>>    text-decoration:underline;}
>> p
>>    {mso-style-priority:99;
>>    mso-margin-top-alt:auto;
>>    margin-right:0in;
>>    mso-margin-bottom-alt:auto;
>>    margin-left:0in;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>>    {mso-style-priority:99;
>>    mso-style-link:"Balloon Text Char";
>>    margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:8.0pt;
>>    font-family:"Tahoma","sans-serif";}
>> span.EmailStyle18
>>    {mso-style-type:personal-reply;
>>    font-family:"Calibri","sans-serif";
>>    color:#1F497D;}
>> span.BalloonTextChar
>>    {mso-style-name:"Balloon Text Char";
>>    mso-style-priority:99;
>>    mso-style-link:"Balloon Text";
>>    font-family:"Tahoma","sans-serif";}
>> .MsoChpDefault
>>    {mso-style-type:export-only;
>>    font-size:10.0pt;}
>> @page WordSection1
>>    {size:8.5in 11.0in;
>>    margin:1.0in 1.0in 1.0in 1.0in;}
>> div.WordSection1
>>    {page:WordSection1;}
>> --></style><!--[if gte mso 9]><xml>
>> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
>> </xml><![endif]--><!--[if gte mso 9]><xml>
>> <o:shapelayout v:ext=3D"edit">
>> <o:idmap v:ext=3D"edit" data=3D"1" />
>> </o:shapelayout></xml><![endif]-->
>> 
>> 
>> <div class=3D"WordSection1">
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>> bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>> :p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>> shes for me on both platforms.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>> r new test program yet.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>> p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <div>
>> <div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>> in 0in">
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>> e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>> </b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>> s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>> a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>> <b>On Behalf Of </b>Jay K<br>
>> <b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>> <b>To:</b> m3devel<br>
>> <b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>> </div>
>> </div>
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>> <div>
>> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>> in-bottom:12.0pt;margin-left:.5in">
>> <span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>> ;This program also doesn't behave as expected, native, nothing to do with wo=
>> w64.
>> <br>
>> &nbsp;Anyone else please confirm: <br>
>> &nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>> &nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>> sp; <br>
>> &nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>> &nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>> &nbsp;I am following up further. <br>
>> &nbsp;Maybe we should get cooperative suspend really going? <br>
>> Thank you.<br>
>> &nbsp;- Jay <br>
>> &nbsp;<br>
>> #include &lt;stdio.h&gt;<br>
>> #include &lt;windows.h&gt;<br>
>> volatile long value;<br>
>> unsigned long __stdcall Thread(PVOID parameter)<br>
>> {<br>
>> &nbsp; while (1)<br>
>> &nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>> &nbsp; return 0;<br>
>> }<br>
>> int __cdecl main()<br>
>> {<br>
>> &nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>> &nbsp; UINT i =3D 0;<br>
>> &nbsp; while (1)<br>
>> &nbsp; {<br>
>> &nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>> &nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>> ;<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>> &nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>> &nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>> &nbsp; }<br>
>> }<o:p></o:p></span></p>
>> </div>
>> </div>
>> 
>> 
>> </div></blockquote></body></html>=
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From rodney_bates at lcwb.coop  Mon Jan 20 17:00:13 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 10:00:13 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD480D.3000101@lcwb.coop>

What does this do?

PROCEDURE Z(VAR x : REFANY) =
   VAR b: BDD.T;
   BEGIN
     b := x;
     x := BDD.Format(b)
   END Z;

The common property of the failing cases seems to be the necessity
of a runtime narrow check when passing x to BDD.Format.

So far, I haven't been able to reproduce the failure on AMD64_LINUX,
with even more irrelevant (one would think) stuff pared out.  I moved
T, Root, and Format into main, removed all but one fields of T and Root,
replaced BDD.New by NEW(Root) and removed Symtab and most of the body
from Format.

Looking at your code, I agree it should work.


On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From rodney_bates at lcwb.coop  Mon Jan 20 18:02:40 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 11:02:40 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD56B0.1050007@lcwb.coop>

Lots more experiments, still no reproduction of the problem on AMD64_LINUX.
The 5 cases in the program below all work.  But earlier versions of this
test program, with bugs, (failure to repeat x:=y before every Z* call)
suggested a hypothesis.

For x := BDD.Format(x)

Perhaps the compiler does something to the LHS x (zeros it out?) before
narrowing and passing the x that is an actual parameter to Format.

If so, it is an implementation bug, as the language says (2.3.1):

   (for v := e)

   "The order of evaluation of v and e is undefined, but e will be evaluated
    before v is updated."

_____________________________________________________________________________________

MODULE Main;

TYPE T = OBJECT l : T; END;

TYPE Root = T OBJECT id : CARDINAL; END;

PROCEDURE Format(x : T) : TEXT =
   BEGIN
     RETURN "Format"
   END Format;

PROCEDURE P() : REFANY =
   BEGIN
     VAR x, y : REFANY;
     BEGIN
       y := NEW ( Root );
       x := y;
       Z1(x);
       x := y;
       Z2(x);
       x := y;
       Z3(x);
       x := y;
       Z4(x);
       x := y;
       Z5(x);
       RETURN x
     END
   END P;

PROCEDURE Z1(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T (b) => x := Format(b)
     END;
   END Z1;

PROCEDURE Z2(VAR x : REFANY) =
   VAR b : T;
   BEGIN
     b := x;
     x := Format(b)
   END Z2;

PROCEDURE Z3(VAR x : REFANY) =
   BEGIN
     x := Format(x)
   END Z3;

PROCEDURE Z4(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T => x := Format (x)
     END;
   END Z4;

PROCEDURE Z5(VAR x : REFANY) =
   BEGIN
     IF ISTYPE (x, T) THEN
       x := Format (x)
     END;
   END Z5;

BEGIN
   EVAL P()
END Main.

______________________________________________________________________________________

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Tue Jan 21 05:18:26 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Mon, 20 Jan 2014 20:18:26 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DD480D.3000101@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
Message-ID: <20140121041826.4D95D1A2080@async.async.caltech.edu>


Were you able to get it to fail on any of your computers?

Your Z fails for me---CM3 and PM3 both.

(579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../src/Main.m3", line 30
***

Abort
(580)truffles:~/bsd/refany/src>cat Mai
Main.m3~  Main.m3   
(580)truffles:~/bsd/refany/src>cat -n Main.m3
     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  (*
    20  PROCEDURE Z(VAR x : REFANY) =
    21    BEGIN
    22      TYPECASE x OF
    23        BDD.T(b) => x := BDD.Format(b)
    24      END;
    25    END Z;
    26  *)
    27  PROCEDURE Z(VAR x : REFANY) =
    28     VAR b: BDD.T;
    29     BEGIN
    30       b := x;
    31       x := BDD.Format(b)
    32     END Z;
    33
    34
    35
    36  BEGIN
    37    Q()
    38  END Main.
(581)truffles:~/bsd/refany/src>
"Rodney M. Bates" writes:
>What does this do?
>
>PROCEDURE Z(VAR x : REFANY) =
>   VAR b: BDD.T;
>   BEGIN
>     b := x;
>     x := BDD.Format(b)
>   END Z;
>
>The common property of the failing cases seems to be the necessity
>of a runtime narrow check when passing x to BDD.Format.
>
>So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>with even more irrelevant (one would think) stuff pared out.  I moved
>T, Root, and Format into main, removed all but one fields of T and Root,
>replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>from Format.
>
>Looking at your code, I agree it should work.
>
>
>On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>> As in the ISTYPE/TYPECASE example, the following works:
>>
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(b)
>>      END;
>>    END Z;
>>
>> (244)rover:~/refany/src>../FreeBSD4/refany
>> a
>>
>>
>> This does NOT work:
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(x)
>>      END;
>>    END Z;
>>
>>
>> (242)rover:~/refany/src>../FreeBSD4/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    NARROW failed
>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>> ***
>>
>>    use option @M3stackdump to get a stack trace
>> Abort
>>
>>
>> Same result PM3 / CM3.
>>
>>       Mika
>>


From rodney_bates at lcwb.coop  Tue Jan 21 17:34:57 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 10:34:57 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEA1B1.2070808@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?

No, the only times I got narrow failures were due to errors
in my trial cases.

I have only tried AMD64_LINUX, but should be able to get this onto a
LINUXLIBC6 system.

>
> Your Z fails for me---CM3 and PM3 both.
>
> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>
>
> ***
> *** runtime error:
> ***    An explicit or implicit NARROW() operation failed.
> ***    file "../src/Main.m3", line 30
> ***

Well that torpedos both my theories.

But it is hard to imagine that bad code generated for an implicit narrow
on an assignment could ever have gone unnoticed so long.  I thought it
might be more believable if it only happened when narrowing an actual
parameter before passing it.

>
> Abort
> (580)truffles:~/bsd/refany/src>cat Mai
> Main.m3~  Main.m3
> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>       1  MODULE Main;
>       2  IMPORT BDD, IO;
>       3
>       4  PROCEDURE P() : REFANY =
>       5    BEGIN
>       6      VAR x : REFANY;
>       7      BEGIN
>       8        x := BDD.New("a");
>       9        Z(x);
>      10        RETURN x
>      11      END
>      12    END P;
>      13
>      14  PROCEDURE Q() =
>      15    BEGIN
>      16      IO.Put(P() & "\n")
>      17    END Q;
>      18
>      19  (*
>      20  PROCEDURE Z(VAR x : REFANY) =
>      21    BEGIN
>      22      TYPECASE x OF
>      23        BDD.T(b) => x := BDD.Format(b)
>      24      END;
>      25    END Z;
>      26  *)
>      27  PROCEDURE Z(VAR x : REFANY) =
>      28     VAR b: BDD.T;
>      29     BEGIN
>      30       b := x;
>      31       x := BDD.Format(b)
>      32     END Z;
>      33
>      34
>      35
>      36  BEGIN
>      37    Q()
>      38  END Main.
> (581)truffles:~/bsd/refany/src>
> "Rodney M. Bates" writes:
>> What does this do?
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    VAR b: BDD.T;
>>    BEGIN
>>      b := x;
>>      x := BDD.Format(b)
>>    END Z;
>>
>> The common property of the failing cases seems to be the necessity
>> of a runtime narrow check when passing x to BDD.Format.
>>
>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>> with even more irrelevant (one would think) stuff pared out.  I moved
>> T, Root, and Format into main, removed all but one fields of T and Root,
>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>from Format.
>>
>> Looking at your code, I agree it should work.
>>
>>
>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>> As in the ISTYPE/TYPECASE example, the following works:
>>>
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(b)
>>>       END;
>>>     END Z;
>>>
>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>> a
>>>
>>>
>>> This does NOT work:
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(x)
>>>       END;
>>>     END Z;
>>>
>>>
>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    NARROW failed
>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>> ***
>>>
>>>     use option @M3stackdump to get a stack trace
>>> Abort
>>>
>>>
>>> Same result PM3 / CM3.
>>>
>>>        Mika
>>>
>



From mika at async.caltech.edu  Tue Jan 21 18:11:20 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 09:11:20 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEA1B1.2070808@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
Message-ID: <20140121171120.F21FF1A2080@async.async.caltech.edu>

Did you try using my bdd.tgz ?

I am getting this error on: 

ancient PM3 on FreeBSD4

head CM3 on AMD64_LINUX

few months old CM3 on ARM_LINUX (Raspberry Pi)

I doubt you will get it on LINUXLIBC6 if you don't get it on the others.

I already tried rearranging the files (the implementation and interfaces
are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
right now).  With no effect.  I don't know what it is about my code that's
tripping up the compiler.

It's also a "decent" BDD package so someone (me I guess, but anyone
else who's interested is welcome to) might want to add it to CM3... :-)
Kind of a pain if you can't print out the debugging info though!
(Actually the workarounds are OK.)

    Mika

"Rodney M. Bates" writes:
>
>
>On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>> Were you able to get it to fail on any of your computers?
>
>No, the only times I got narrow failures were due to errors
>in my trial cases.
>
>I have only tried AMD64_LINUX, but should be able to get this onto a
>LINUXLIBC6 system.
>
>>
>> Your Z fails for me---CM3 and PM3 both.
>>
>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    An explicit or implicit NARROW() operation failed.
>> ***    file "../src/Main.m3", line 30
>> ***
>
>Well that torpedos both my theories.
>
>But it is hard to imagine that bad code generated for an implicit narrow
>on an assignment could ever have gone unnoticed so long.  I thought it
>might be more believable if it only happened when narrowing an actual
>parameter before passing it.
>
>>
>> Abort
>> (580)truffles:~/bsd/refany/src>cat Mai
>> Main.m3~  Main.m3
>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>       1  MODULE Main;
>>       2  IMPORT BDD, IO;
>>       3
>>       4  PROCEDURE P() : REFANY =
>>       5    BEGIN
>>       6      VAR x : REFANY;
>>       7      BEGIN
>>       8        x := BDD.New("a");
>>       9        Z(x);
>>      10        RETURN x
>>      11      END
>>      12    END P;
>>      13
>>      14  PROCEDURE Q() =
>>      15    BEGIN
>>      16      IO.Put(P() & "\n")
>>      17    END Q;
>>      18
>>      19  (*
>>      20  PROCEDURE Z(VAR x : REFANY) =
>>      21    BEGIN
>>      22      TYPECASE x OF
>>      23        BDD.T(b) => x := BDD.Format(b)
>>      24      END;
>>      25    END Z;
>>      26  *)
>>      27  PROCEDURE Z(VAR x : REFANY) =
>>      28     VAR b: BDD.T;
>>      29     BEGIN
>>      30       b := x;
>>      31       x := BDD.Format(b)
>>      32     END Z;
>>      33
>>      34
>>      35
>>      36  BEGIN
>>      37    Q()
>>      38  END Main.
>> (581)truffles:~/bsd/refany/src>
>> "Rodney M. Bates" writes:
>>> What does this do?
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>    VAR b: BDD.T;
>>>    BEGIN
>>>      b := x;
>>>      x := BDD.Format(b)
>>>    END Z;
>>>
>>> The common property of the failing cases seems to be the necessity
>>> of a runtime narrow check when passing x to BDD.Format.
>>>
>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>from Format.
>>>
>>> Looking at your code, I agree it should work.
>>>
>>>
>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(b)
>>>>       END;
>>>>     END Z;
>>>>
>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>> a
>>>>
>>>>
>>>> This does NOT work:
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(x)
>>>>       END;
>>>>     END Z;
>>>>
>>>>
>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    NARROW failed
>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>> ***
>>>>
>>>>     use option @M3stackdump to get a stack trace
>>>> Abort
>>>>
>>>>
>>>> Same result PM3 / CM3.
>>>>
>>>>        Mika
>>>>
>>


From rodney_bates at lcwb.coop  Tue Jan 21 18:23:41 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 11:23:41 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEAD1D.8010602@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?
>

My test case with 5 versions of Z, and your complete one that you posted
(bdd.tgz) both silently complete with no error messages on LINUXLIBC6,
using the head compiler and libraries.

bdd.tgz also works fine on AMD64_LINUX.

On yours, I built and shipped in bdd/src, then built in bdd/test/src
and ran LINUXLIBC6/bddtest, which I presume was the right procedure.

How about compiling a couple of failing Z versions on a machine where
they fail, with -keep, then posting the .mc, .ms, and .mo files and corresponding
source (only for the Zs is probably enough).


>>>        Mika
>>>
>



From rodney_bates at lcwb.coop  Tue Jan 21 19:02:09 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 12:02:09 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121171120.F21FF1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
Message-ID: <52DEB621.7080706@lcwb.coop>



On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
> Did you try using my bdd.tgz ?
>
> I am getting this error on:
>
> ancient PM3 on FreeBSD4
>
> head CM3 on AMD64_LINUX

Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
slightly different in size when I recompiled, but I think there is an explanation
for that.

I can't build bddtest with the release compiler, because it needs caltech_parser,
and that won't build in the release.  This combination of [non]failures seems
pretty strange.  I am also wondering about a problem with type checks in the
runtime.


>
> few months old CM3 on ARM_LINUX (Raspberry Pi)
>
> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>
> I already tried rearranging the files (the implementation and interfaces
> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
> right now).  With no effect.  I don't know what it is about my code that's
> tripping up the compiler.
>
> It's also a "decent" BDD package so someone (me I guess, but anyone
> else who's interested is welcome to) might want to add it to CM3... :-)
> Kind of a pain if you can't print out the debugging info though!
> (Actually the workarounds are OK.)
>
>      Mika
>
> "Rodney M. Bates" writes:
>>
>>
>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>> Were you able to get it to fail on any of your computers?
>>
>> No, the only times I got narrow failures were due to errors
>> in my trial cases.
>>
>> I have only tried AMD64_LINUX, but should be able to get this onto a
>> LINUXLIBC6 system.
>>
>>>
>>> Your Z fails for me---CM3 and PM3 both.
>>>
>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    An explicit or implicit NARROW() operation failed.
>>> ***    file "../src/Main.m3", line 30
>>> ***
>>
>> Well that torpedos both my theories.
>>
>> But it is hard to imagine that bad code generated for an implicit narrow
>> on an assignment could ever have gone unnoticed so long.  I thought it
>> might be more believable if it only happened when narrowing an actual
>> parameter before passing it.
>>
>>>
>>> Abort
>>> (580)truffles:~/bsd/refany/src>cat Mai
>>> Main.m3~  Main.m3
>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>        1  MODULE Main;
>>>        2  IMPORT BDD, IO;
>>>        3
>>>        4  PROCEDURE P() : REFANY =
>>>        5    BEGIN
>>>        6      VAR x : REFANY;
>>>        7      BEGIN
>>>        8        x := BDD.New("a");
>>>        9        Z(x);
>>>       10        RETURN x
>>>       11      END
>>>       12    END P;
>>>       13
>>>       14  PROCEDURE Q() =
>>>       15    BEGIN
>>>       16      IO.Put(P() & "\n")
>>>       17    END Q;
>>>       18
>>>       19  (*
>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>       21    BEGIN
>>>       22      TYPECASE x OF
>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>       24      END;
>>>       25    END Z;
>>>       26  *)
>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>       28     VAR b: BDD.T;
>>>       29     BEGIN
>>>       30       b := x;
>>>       31       x := BDD.Format(b)
>>>       32     END Z;
>>>       33
>>>       34
>>>       35
>>>       36  BEGIN
>>>       37    Q()
>>>       38  END Main.
>>> (581)truffles:~/bsd/refany/src>
>>> "Rodney M. Bates" writes:
>>>> What does this do?
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     VAR b: BDD.T;
>>>>     BEGIN
>>>>       b := x;
>>>>       x := BDD.Format(b)
>>>>     END Z;
>>>>
>>>> The common property of the failing cases seems to be the necessity
>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>
>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>> >from Format.
>>>>
>>>> Looking at your code, I agree it should work.
>>>>
>>>>
>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>> a
>>>>>
>>>>>
>>>>> This does NOT work:
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>>
>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>
>>>>>
>>>>> ***
>>>>> *** runtime error:
>>>>> ***    NARROW failed
>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>> ***
>>>>>
>>>>>      use option @M3stackdump to get a stack trace
>>>>> Abort
>>>>>
>>>>>
>>>>> Same result PM3 / CM3.
>>>>>
>>>>>         Mika
>>>>>
>>>
>



From mika at async.caltech.edu  Tue Jan 21 19:08:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 10:08:06 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEB621.7080706@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
Message-ID: <20140121180806.D75A61A208F@async.async.caltech.edu>

The only thing bddtest uses from caltech_parser is Debug.Out.  You can just take out those calls or replace them with IO.Put.

But again bddtest is not the program that fails... it's the "refany" program (now in a sibling directory to bddtest)

   Mika

"Rodney M. Bates" writes:
>
>
>On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>> Did you try using my bdd.tgz ?
>>
>> I am getting this error on:
>>
>> ancient PM3 on FreeBSD4
>>
>> head CM3 on AMD64_LINUX
>
>Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
>a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
>but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
>slightly different in size when I recompiled, but I think there is an explanation
>for that.
>
>I can't build bddtest with the release compiler, because it needs caltech_parser,
>and that won't build in the release.  This combination of [non]failures seems
>pretty strange.  I am also wondering about a problem with type checks in the
>runtime.
>
>
>>
>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>
>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>>
>> I already tried rearranging the files (the implementation and interfaces
>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>> right now).  With no effect.  I don't know what it is about my code that's
>> tripping up the compiler.
>>
>> It's also a "decent" BDD package so someone (me I guess, but anyone
>> else who's interested is welcome to) might want to add it to CM3... :-)
>> Kind of a pain if you can't print out the debugging info though!
>> (Actually the workarounds are OK.)
>>
>>      Mika
>>
>> "Rodney M. Bates" writes:
>>>
>>>
>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>> Were you able to get it to fail on any of your computers?
>>>
>>> No, the only times I got narrow failures were due to errors
>>> in my trial cases.
>>>
>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>> LINUXLIBC6 system.
>>>
>>>>
>>>> Your Z fails for me---CM3 and PM3 both.
>>>>
>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    An explicit or implicit NARROW() operation failed.
>>>> ***    file "../src/Main.m3", line 30
>>>> ***
>>>
>>> Well that torpedos both my theories.
>>>
>>> But it is hard to imagine that bad code generated for an implicit narrow
>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>> might be more believable if it only happened when narrowing an actual
>>> parameter before passing it.
>>>
>>>>
>>>> Abort
>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>> Main.m3~  Main.m3
>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>        1  MODULE Main;
>>>>        2  IMPORT BDD, IO;
>>>>        3
>>>>        4  PROCEDURE P() : REFANY =
>>>>        5    BEGIN
>>>>        6      VAR x : REFANY;
>>>>        7      BEGIN
>>>>        8        x := BDD.New("a");
>>>>        9        Z(x);
>>>>       10        RETURN x
>>>>       11      END
>>>>       12    END P;
>>>>       13
>>>>       14  PROCEDURE Q() =
>>>>       15    BEGIN
>>>>       16      IO.Put(P() & "\n")
>>>>       17    END Q;
>>>>       18
>>>>       19  (*
>>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>>       21    BEGIN
>>>>       22      TYPECASE x OF
>>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>>       24      END;
>>>>       25    END Z;
>>>>       26  *)
>>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>>       28     VAR b: BDD.T;
>>>>       29     BEGIN
>>>>       30       b := x;
>>>>       31       x := BDD.Format(b)
>>>>       32     END Z;
>>>>       33
>>>>       34
>>>>       35
>>>>       36  BEGIN
>>>>       37    Q()
>>>>       38  END Main.
>>>> (581)truffles:~/bsd/refany/src>
>>>> "Rodney M. Bates" writes:
>>>>> What does this do?
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>     VAR b: BDD.T;
>>>>>     BEGIN
>>>>>       b := x;
>>>>>       x := BDD.Format(b)
>>>>>     END Z;
>>>>>
>>>>> The common property of the failing cases seems to be the necessity
>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>
>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>> >from Format.
>>>>>
>>>>> Looking at your code, I agree it should work.
>>>>>
>>>>>
>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>> a
>>>>>>
>>>>>>
>>>>>> This does NOT work:
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>>
>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>
>>>>>>
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    NARROW failed
>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>> ***
>>>>>>
>>>>>>      use option @M3stackdump to get a stack trace
>>>>>> Abort
>>>>>>
>>>>>>
>>>>>> Same result PM3 / CM3.
>>>>>>
>>>>>>         Mika
>>>>>>
>>>>
>>


From mika at async.caltech.edu  Tue Jan 21 20:23:28 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 11:23:28 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
	<20140121180806.D75A61A208F@async.async.caltech.edu>
	<07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
Message-ID: <20140121192328.92DDB1A208F@async.async.caltech.edu>

How do you get them into text form?  They look binary...

I copied the whole directory bdd (and all subdirs) to 

http://rover.gcapltd.com/~mika/bdd/

Antony Hosking writes:
>Please post .io / .mo in text form.
>
>Sent from my iPad
>
>> On Jan 21, 2014, at 1:08 PM, mika at async.caltech.edu wrote:
>>=20
>> The only thing bddtest uses from caltech_parser is Debug.Out.  You can jus=
>t take out those calls or replace them with IO.Put.
>>=20
>> But again bddtest is not the program that fails... it's the "refany" progr=
>am (now in a sibling directory to bddtest)
>>=20
>>   Mika
>>=20
>> "Rodney M. Bates" writes:
>>>=20
>>>=20
>>>> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>>>> Did you try using my bdd.tgz ?
>>>>=20
>>>> I am getting this error on:
>>>>=20
>>>> ancient PM3 on FreeBSD4
>>>>=20
>>>> head CM3 on AMD64_LINUX
>>>=20
>>> Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX s=
>ystem with
>>> a freshly updated head.  Hacking around, I tried to use the binaries in b=
>dd.tgz,
>>> but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files=
> are
>>> slightly different in size when I recompiled, but I think there is an exp=
>lanation
>>> for that.
>>>=20
>>> I can't build bddtest with the release compiler, because it needs caltech=
>_parser,
>>> and that won't build in the release.  This combination of [non]failures s=
>eems
>>> pretty strange.  I am also wondering about a problem with type checks in t=
>he
>>> runtime.
>>>=20
>>>=20
>>>>=20
>>>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>>>=20
>>>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.=
>
>>>>=20
>>>> I already tried rearranging the files (the implementation and interfaces=
>
>>>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>>>> right now).  With no effect.  I don't know what it is about my code that=
>'s
>>>> tripping up the compiler.
>>>>=20
>>>> It's also a "decent" BDD package so someone (me I guess, but anyone
>>>> else who's interested is welcome to) might want to add it to CM3... :-)
>>>> Kind of a pain if you can't print out the debugging info though!
>>>> (Actually the workarounds are OK.)
>>>>=20
>>>>     Mika
>>>>=20
>>>> "Rodney M. Bates" writes:
>>>>>=20
>>>>>=20
>>>>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>>>> Were you able to get it to fail on any of your computers?
>>>>>=20
>>>>> No, the only times I got narrow failures were due to errors
>>>>> in my trial cases.
>>>>>=20
>>>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>>>> LINUXLIBC6 system.
>>>>>=20
>>>>>>=20
>>>>>> Your Z fails for me---CM3 and PM3 both.
>>>>>>=20
>>>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>>>=20
>>>>>>=20
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    An explicit or implicit NARROW() operation failed.
>>>>>> ***    file "../src/Main.m3", line 30
>>>>>> ***
>>>>>=20
>>>>> Well that torpedos both my theories.
>>>>>=20
>>>>> But it is hard to imagine that bad code generated for an implicit narro=
>w
>>>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>>>> might be more believable if it only happened when narrowing an actual
>>>>> parameter before passing it.
>>>>>=20
>>>>>>=20
>>>>>> Abort
>>>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>>>> Main.m3~  Main.m3
>>>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>>>       1  MODULE Main;
>>>>>>       2  IMPORT BDD, IO;
>>>>>>       3
>>>>>>       4  PROCEDURE P() : REFANY =3D
>>>>>>       5    BEGIN
>>>>>>       6      VAR x : REFANY;
>>>>>>       7      BEGIN
>>>>>>       8        x :=3D BDD.New("a");
>>>>>>       9        Z(x);
>>>>>>      10        RETURN x
>>>>>>      11      END
>>>>>>      12    END P;
>>>>>>      13
>>>>>>      14  PROCEDURE Q() =3D
>>>>>>      15    BEGIN
>>>>>>      16      IO.Put(P() & "\n")
>>>>>>      17    END Q;
>>>>>>      18
>>>>>>      19  (*
>>>>>>      20  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      21    BEGIN
>>>>>>      22      TYPECASE x OF
>>>>>>      23        BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>      24      END;
>>>>>>      25    END Z;
>>>>>>      26  *)
>>>>>>      27  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      28     VAR b: BDD.T;
>>>>>>      29     BEGIN
>>>>>>      30       b :=3D x;
>>>>>>      31       x :=3D BDD.Format(b)
>>>>>>      32     END Z;
>>>>>>      33
>>>>>>      34
>>>>>>      35
>>>>>>      36  BEGIN
>>>>>>      37    Q()
>>>>>>      38  END Main.
>>>>>> (581)truffles:~/bsd/refany/src>
>>>>>> "Rodney M. Bates" writes:
>>>>>>> What does this do?
>>>>>>>=20
>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>    VAR b: BDD.T;
>>>>>>>    BEGIN
>>>>>>>      b :=3D x;
>>>>>>>      x :=3D BDD.Format(b)
>>>>>>>    END Z;
>>>>>>>=20
>>>>>>> The common property of the failing cases seems to be the necessity
>>>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>>>=20
>>>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>>>> with even more irrelevant (one would think) stuff pared out.  I moved=
>
>>>>>>> T, Root, and Format into main, removed all but one fields of T and Ro=
>ot,
>>>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body=
>
>>>>>>> from Format.
>>>>>>>=20
>>>>>>> Looking at your code, I agree it should work.
>>>>>>>=20
>>>>>>>=20
>>>>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>> a
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> This does NOT work:
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(x)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> ***
>>>>>>>> *** runtime error:
>>>>>>>> ***    NARROW failed
>>>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>>>> ***
>>>>>>>>=20
>>>>>>>>     use option @M3stackdump to get a stack trace
>>>>>>>> Abort
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Same result PM3 / CM3.
>>>>>>>>=20
>>>>>>>>        Mika
>>>>=20


From rodney_bates at lcwb.coop  Wed Jan 22 20:46:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Wed, 22 Jan 2014 13:46:55 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52E0202F.3020307@lcwb.coop>

Some more info:

The front end is generating incorrect narrow code, only
where the type T is an unrevealed <: REFANY.  When T and Root
are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
and implicit narrow in an assignment.

When opaque, for explicit and implicit narrow, it simply compares
the typecode of x to a single typecode (no doubt for T) and demands
they be equal.  In this case, the actual allocated type of x is Root,
a proper subtype of T.

For ISTYPE, it does call CheckIsType even when the types are opaque.

The fact that it happens only when the types are opaque lends
credibility to the fact that this bug has gone unnoticed at least
as far back as PM3.

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Wed Jan 22 21:47:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 22 Jan 2014 12:47:49 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E0202F.3020307@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
Message-ID: <20140122204749.C724E1A208F@async.async.caltech.edu>


Aha so declaring the type as <: ROOT will probably work as a workaround?

Thank you for looking into this.

"Rodney M. Bates" writes:
>Some more info:
>
>The front end is generating incorrect narrow code, only
>where the type T is an unrevealed <: REFANY.  When T and Root
>are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>and implicit narrow in an assignment.
>
>When opaque, for explicit and implicit narrow, it simply compares
>the typecode of x to a single typecode (no doubt for T) and demands
>they be equal.  In this case, the actual allocated type of x is Root,
>a proper subtype of T.
>
>For ISTYPE, it does call CheckIsType even when the types are opaque.
>
>The fact that it happens only when the types are opaque lends
>credibility to the fact that this bug has gone unnoticed at least
>as far back as PM3.


From jay.krell at cornell.edu  Thu Jan 23 08:51:58 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 07:51:58 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/046e93c5/attachment-0001.html>

From rcolebur at SCIRES.COM  Thu Jan 23 12:59:34 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 11:59:34 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
Message-ID: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:

  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/ab6f0152/attachment-0001.html>

From rodney_bates at lcwb.coop  Thu Jan 23 16:25:59 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 23 Jan 2014 09:25:59 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140122204749.C724E1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
Message-ID: <52E13487.9090504@lcwb.coop>



On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
> Aha so declaring the type as <: ROOT will probably work as a workaround?
>

I'm not sure whether that would do it or not.  In my pared-down case, I had
moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
the failures go away.

Until it is fixed, I think the best workaround is TYPECASE with a binding:

TYPECASE x
OF BDD.T(b) =>
   Format(b), etc.

This is known to be working now.  You've probably thought about this, but
this way requires only a single runtime type test.  Either TYPECASE without
the binding of b, or ISTYPE, will require an additional recheck of the same
thing when the narrow, implicit or explicit, happens.  I doubt even a
pretty good optimizing compiler could remove this.  For ISTYPE, it would have
to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
relation between CheckIsType and ScanTypecase.

> Thank you for looking into this.
>
> "Rodney M. Bates" writes:
>> Some more info:
>>
>> The front end is generating incorrect narrow code, only
>> where the type T is an unrevealed <: REFANY.  When T and Root
>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>> and implicit narrow in an assignment.
>>
>> When opaque, for explicit and implicit narrow, it simply compares
>> the typecode of x to a single typecode (no doubt for T) and demands
>> they be equal.  In this case, the actual allocated type of x is Root,
>> a proper subtype of T.
>>
>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>
>> The fact that it happens only when the types are opaque lends
>> credibility to the fact that this bug has gone unnoticed at least
>> as far back as PM3.
>



From hosking at cs.purdue.edu  Thu Jan 23 17:51:02 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 23 Jan 2014 11:51:02 -0500
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
Message-ID: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> Jay,
> 
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
> 
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
> 
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
> 
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
> 
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
> 
>>   There is a behavior/bug in wow64.  
>>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
>> 
>> 
>> 
>>   SuspendThread / GetThreadContext work like this:  
>> 
>> 
>>   32bit processes consist almost entirely of 32bit code.  
>>   There is a small amount of 64bit code.  
>> 
>> 
>>   If you suspend while running 32bit code, GetThreadContext works.  
>>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
>> 
>> 
>>   64bit code is run en route to syscalls.  
>>   For example you call:  
>>     1 kernel32!Sleep   
>>     2 it calls 32bit NtDelayExecution   
>>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>>     4 which calls native NtDelayExecuation   
>> 
>>  
>>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>>     You can step through it very easily in a debugger. Really. 
>>   Where GetThreadContext knows where to get it.  
>>   The problem is that saving context is not atomic.  
>>   You can suspend while saving context.  
>> 
>> 
>>   What to do? 
>> 
>> 
>>   scratch/wow64stack contains a program that detects the bug.
>>   I believe it is the basis of a workaround for the bug.   
>> 
>> 
>>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>>   not only functions that use exception handling, but also functions that call "extern" functions  
>>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>>   use that, if it isn't zero. 
>>   Normally it will be zero.  
>>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>>   That is why it isn't a set/set-zero pattern (like in the test program).   
>> 
>> 
>>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
>> 
>> 
>>   Rejected counter proposal: 
>>    Don't suspend/gc when running a syscall.  
>>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
>> 
>> 
>>  Possible augmentation: if running native, short circuit most of this 
>> 
>> 
>>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
>> 
>> 
>>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
>> 
>> 
>>   Performance impact: hypothetically large but probably not noticable. 
>> 
>> 
>>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>>   We could further augment pragmas to discern them. 
>> 
>> 
>>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
>> 
>> 
>> 
>>   Agreed? I'll make the compiler change? 
>> 
>> 
>> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
>> 
>> 
>> Eventually cooperative suspend will cause this to fall away as a problem..
>> 
>> 
>>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/8a7dd0a9/attachment-0001.html>

From jay.krell at cornell.edu  Thu Jan 23 21:03:09 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 20:03:09 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>,
	<COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
Message-ID: <COL130-W24D43515B172BD96CF08F6E6A60@phx.gbl>

Syscalls are "doubly wrapped".
Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention.
Lesser known, there are also some syscalls in gdi32.dll and user32.dll.
None of them are public interfaces.
They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll.
For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change.
Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work.

There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall.

Bottom line is though, one isn't really supposed to know where the syscalls are.

 - Jay

From: jay.krell at cornell.edu
To: hosking at cs.purdue.edu; rcolebur at scires.com
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
Date: Thu, 23 Jan 2014 19:57:27 +0000




There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/7b51ac64/attachment-0001.html>

From jay.krell at cornell.edu  Thu Jan 23 20:57:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 19:57:27 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
Message-ID: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5f6fec82/attachment-0001.html>

From rcolebur at SCIRES.COM  Thu Jan 23 23:23:43 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

When you say native 32/64-bit "should be ok", you really mean ok with respect to the SYSWOW64 problem you've identified.

Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay
________________________________
From: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>; jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484






On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:
  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/c534e985/attachment-0001.html>

From jay.krell at cornell.edu  Fri Jan 24 00:09:50 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 23:09:50 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>

Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?









Jay:
 
When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 

 
Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Thursday, January 23, 2014 2:57 PM

To: Tony; Coleburn, Randy

Cc: m3devel

Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?


 


There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.

Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.



 - Jay





From:
hosking at cs.purdue.edu

Date: Thu, 23 Jan 2014 11:51:02 -0500

To: rcolebur at SCIRES.COM

CC: m3devel at elegosoft.com; 
jay.krell at cornell.edu

Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?



Yes, agreed.

We need to address the native thread problem on all platforms (both pthread and windows).


 


However, this particular problem is pernicious if we need to save M3 stack state at all external calls.


 


Are there any linker tricks for Windows one can use to wrap only system calls?


 


I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,
  This approach is used for all modern Java implementations that I am aware of.


 


It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread
 state.


 




Antony Hosking | Associate
 Professor | Computer Science | Purdue University


305 N. University Street | West Lafayette | IN 47907 | USA


Mobile +1
 765 427 5484


 














 


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


Jay,


 


If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 


 


Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both
 32 & 64 bit platforms. 


 


Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 


 


Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?


 


--Randy



Sent from my iPhone





On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:



  There is a behavior/bug in wow64.  

  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.







  SuspendThread / GetThreadContext work like this:  





  32bit processes consist almost entirely of 32bit code.  

  There is a small amount of 64bit code.  





  If you suspend while running 32bit code, GetThreadContext works.  

  if you suspend while running 64bit code, GetThreadContext usually but not always works.  





  64bit code is run en route to syscalls.  

  For example you call:  

    1 kernel32!Sleep   

    2 it calls 32bit NtDelayExecution   

    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 

    4 which calls native NtDelayExecuation   



 

  In between 2 and 3, within 64bit code, the 32bit context is saved.  

    You can step through it very easily in a debugger. Really. 

  Where GetThreadContext knows where to get it.  

  The problem is that saving context is not atomic.  

  You can suspend while saving context.  





  What to do? 





  scratch/wow64stack contains a program that detects the bug.

  I believe it is the basis of a workaround for the bug.   





  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 

  not only functions that use exception handling, but also functions that call "extern" functions  

  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection

  use that, if it isn't zero. 

  Normally it will be zero.  

  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  

  That is why it isn't a set/set-zero pattern (like in the test program).   





  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 





  Rejected counter proposal: 

   Don't suspend/gc when running a syscall.  

   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  





 Possible augmentation: if running native, short circuit most of this 





  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 

   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 





   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 

   until/unless there is another 32bit platform runable on some similar 64bit platforms. 





  Performance impact: hypothetically large but probably not noticable. 





  Furthe refinements: It isn't extern/native code per se, it is syscalls.

  We could further augment pragmas to discern them. 





  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 

  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 

  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 







  Agreed? I'll make the compiler change? 





Oh, also, not just stack pointer, but other registers, at least non-volatiles?





Eventually cooperative suspend will cause this to fall away as a problem..





 - Jay





 



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5aed95c6/attachment-0001.html>

From adacore at marino.st  Fri Jan 24 00:42:01 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 00:42:01 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBCDBA.8030809@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>
Message-ID: <52E1A8C9.3010903@marino.st>

On 1/19/2014 14:06, John Marino wrote:
> On 1/19/2014 13:50, Jay K wrote:
>> And there are advantages not yet implemented, in particular, efficient
>> exception handling by generating C++.
> 
> Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> dl_iterate_phdr which gcc uses for efficient zero-cost exception
> handling.  That might come in handy here (I implemented it in
> DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> 
> 
>> Your patches should apply fairly cleanly to any new checkout.
>> I am willing to through them and apply them to CVS if you want --
>> depends on if you are ok with my stealing your credit, vs. you want to
>> commit them yourselves.
> 
> I would love it if you did this - I'm happy with a mention in the commit
> message.  I just attached a tarball of all the patches I'm using right
> now.  Some of these changes are specifically for the cross compiler (I'm
> think of the /scripts changes mostly) but most are valid assuming they
> still apply to the head of the repo.  Please take as much as you want
> with my gratitude.
> 
>> I already skimmed them and they all look easy.
>> DragonflyBSD is basically FreeBSD by another name.
> 
> That's not an accident.  As a DragonFly developer that mostly works on
> userland, I've been converging FreeBSD and DragonFly again.  They
> drifted apart after 10 years, but I thought it better to keep the
> userlands as similar as possible for 3rd party applications.
> 
>> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
>> But the ABI is almost the same -- heck, for the most part,
>> Linux==NetBSD==FreeBSD==OpenBSD.
>> Sure, they might be implemented differently, but they are highly source
>> compatible and every highly object code compatible.
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining, and it was built by the latest patches.
>> Agreed, we are both missing something simple.
>> Usually this is the error you get when you don't upgrade
>> Target.i3/Target.m3.
>> If you want to press on with the current approach and ignore my advise,
> 
> I'm not ignoring it; it's more like I'm not quite ready to give up since
> I'm so close.  I think I could actually get these libs and cm3 to build
> on DragonFly if I used AMD64_FREEBSD as the target with the
> cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> I'd then get stuck on the DragonFly side trying to compile cm3 with
> itself, theoretically I should get the same error there.
> 
> 
>> Ugh. I saw something about that in your diffs. Definitely we intend to
>> link statically.
>> I actually went to the trouble of removing use of gmp/mpfr/mpc in
>> current source.
>> They are used for actually very little and aren't worth it.
> 

Hi Jay,
If you get those patches committed and publish an official snapshot
(compressed tarball) with those changes in it, then I'll update the
FreeBSD port to that snapshot.

If new patches are required, say to convert FreeBSD to use the C
backend, I'll feed those back too.

Then I'll try porting DragonFly again, your way.
Thanks,
John





From jay.krell at cornell.edu  Fri Jan 24 09:45:39 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 08:45:39 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E1A8C9.3010903@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
Message-ID: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>

sorry, I'm delayed..so much to do..
I commited the networking fix..
The m3core/unix one I think we already have.

Later...
 - Jay

> Date: Fri, 24 Jan 2014 00:42:01 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 14:06, John Marino wrote:
> > On 1/19/2014 13:50, Jay K wrote:
> >> And there are advantages not yet implemented, in particular, efficient
> >> exception handling by generating C++.
> > 
> > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> > dl_iterate_phdr which gcc uses for efficient zero-cost exception
> > handling.  That might come in handy here (I implemented it in
> > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> > 
> > 
> >> Your patches should apply fairly cleanly to any new checkout.
> >> I am willing to through them and apply them to CVS if you want --
> >> depends on if you are ok with my stealing your credit, vs. you want to
> >> commit them yourselves.
> > 
> > I would love it if you did this - I'm happy with a mention in the commit
> > message.  I just attached a tarball of all the patches I'm using right
> > now.  Some of these changes are specifically for the cross compiler (I'm
> > think of the /scripts changes mostly) but most are valid assuming they
> > still apply to the head of the repo.  Please take as much as you want
> > with my gratitude.
> > 
> >> I already skimmed them and they all look easy.
> >> DragonflyBSD is basically FreeBSD by another name.
> > 
> > That's not an accident.  As a DragonFly developer that mostly works on
> > userland, I've been converging FreeBSD and DragonFly again.  They
> > drifted apart after 10 years, but I thought it better to keep the
> > userlands as similar as possible for 3rd party applications.
> > 
> >> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> >> But the ABI is almost the same -- heck, for the most part,
> >> Linux==NetBSD==FreeBSD==OpenBSD.
> >> Sure, they might be implemented differently, but they are highly source
> >> compatible and every highly object code compatible.
> >>> I must be misunderstanding what you mean by "upgrade". It's the
> >>> cross-compiler that's complaining, and it was built by the latest patches.
> >> Agreed, we are both missing something simple.
> >> Usually this is the error you get when you don't upgrade
> >> Target.i3/Target.m3.
> >> If you want to press on with the current approach and ignore my advise,
> > 
> > I'm not ignoring it; it's more like I'm not quite ready to give up since
> > I'm so close.  I think I could actually get these libs and cm3 to build
> > on DragonFly if I used AMD64_FREEBSD as the target with the
> > cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> > I'd then get stuck on the DragonFly side trying to compile cm3 with
> > itself, theoretically I should get the same error there.
> > 
> > 
> >> Ugh. I saw something about that in your diffs. Definitely we intend to
> >> link statically.
> >> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> >> current source.
> >> They are used for actually very little and aren't worth it.
> > 
> 
> Hi Jay,
> If you get those patches committed and publish an official snapshot
> (compressed tarball) with those changes in it, then I'll update the
> FreeBSD port to that snapshot.
> 
> If new patches are required, say to convert FreeBSD to use the C
> backend, I'll feed those back too.
> 
> Then I'll try porting DragonFly again, your way.
> Thanks,
> John
> 
> 
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9f8f3f93/attachment-0001.html>

From adacore at marino.st  Fri Jan 24 10:11:02 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 10:11:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>, <52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
Message-ID: <52E22E26.10402@marino.st>

On 1/24/2014 09:45, Jay K wrote:
> sorry, I'm delayed..so much to do..
> I commited the networking fix..
> The m3core/unix one I think we already have.
> 
> Later...
>  - Jay


Right, but there's no downloadable source snapshot.  And if the
DragonFly support isn't in there, I'll have to carry all those patches
too.  So my goal is to have a few patches as possible.

Anyway, I have to base the port on something -- pulling latest SVN is
rarely done, and CVS is never done.  Some github projects don't make
releases and needed to be pulled directly, but this requires first git
to be built (lots of dependencies) and then github tweaks their service
and everything fails.  So I'm a big believer in static compressed
tarballs.

Which means I can't update FreeBSD cm3 version until one exists.
theoretically I can package one myself, but it looks better if it's
"official" and comes from one of the known m3 websites.

Thanks,
John


From jay.krell at cornell.edu  Fri Jan 24 10:59:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 09:59:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E22E26.10402@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>,
	<52E22E26.10402@marino.st>
Message-ID: <COL130-W58042B8075CEF3E2C8165CE6A10@phx.gbl>

Please be patient. We'll get a snapshot, soon. A release, unclear.

  - Jay

> Date: Fri, 24 Jan 2014 10:11:02 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/24/2014 09:45, Jay K wrote:
> > sorry, I'm delayed..so much to do..
> > I commited the networking fix..
> > The m3core/unix one I think we already have.
> > 
> > Later...
> >  - Jay
> 
> 
> Right, but there's no downloadable source snapshot.  And if the
> DragonFly support isn't in there, I'll have to carry all those patches
> too.  So my goal is to have a few patches as possible.
> 
> Anyway, I have to base the port on something -- pulling latest SVN is
> rarely done, and CVS is never done.  Some github projects don't make
> releases and needed to be pulled directly, but this requires first git
> to be built (lots of dependencies) and then github tweaks their service
> and everything fails.  So I'm a big believer in static compressed
> tarballs.
> 
> Which means I can't update FreeBSD cm3 version until one exists.
> theoretically I can package one myself, but it looks better if it's
> "official" and comes from one of the known m3 websites.
> 
> Thanks,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9ee26689/attachment-0001.html>

From jay.krell at cornell.edu  Fri Jan 24 11:38:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 10:38:01 +0000
Subject: [M3devel] garbage collector compacting and volatile?
Message-ID: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>

Is our garbage collector compacting?


In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?
Update references anywhere? Just the stack? Also context?


There are tradeoffs either way.


If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?


If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.


If we read and write registers in the collector, then nothing has to be volatile.


Should this all be parameters passed to the backend?
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.


If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.


Thank you,
 - Jay

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/e86b30bc/attachment-0001.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:24:35 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:24:35 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
Message-ID: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.
Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:

> Yes. Agreed.
> They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
> I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
>  
>  
>  - Jay
> 
>  
> From: rcolebur at SCIRES.COM
> To: m3devel at elegosoft.com
> Date: Thu, 23 Jan 2014 22:23:43 +0000
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> Jay:
>  
> When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 
>  
> Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Thursday, January 23, 2014 2:57 PM
> To: Tony; Coleburn, Randy
> Cc: m3devel
> Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
>  
> There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
> Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.
> 
>  - Jay
> From: hosking at cs.purdue.edu
> Date: Thu, 23 Jan 2014 11:51:02 -0500
> To: rcolebur at SCIRES.COM
> CC: m3devel at elegosoft.com; jay.krell at cornell.edu
> Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
> 
> Yes, agreed.
> We need to address the native thread problem on all platforms (both pthread and windows).
>  
> However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
>  
> Are there any linker tricks for Windows one can use to wrap only system calls?
>  
> I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
>  
> It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.
>  
> Antony Hosking | Associate Professor | Computer Science | Purdue University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Mobile +1 765 427 5484
>  
> 
> 
> 
> 
>  
> On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:
>  
> Jay,
>  
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
>  
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
>  
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
>  
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
>  
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
>   There is a behavior/bug in wow64.  
>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
> 
> 
> 
>   SuspendThread / GetThreadContext work like this:  
> 
> 
>   32bit processes consist almost entirely of 32bit code.  
>   There is a small amount of 64bit code.  
> 
> 
>   If you suspend while running 32bit code, GetThreadContext works.  
>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
> 
> 
>   64bit code is run en route to syscalls.  
>   For example you call:  
>     1 kernel32!Sleep   
>     2 it calls 32bit NtDelayExecution   
>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>     4 which calls native NtDelayExecuation   
> 
>  
>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>     You can step through it very easily in a debugger. Really. 
>   Where GetThreadContext knows where to get it.  
>   The problem is that saving context is not atomic.  
>   You can suspend while saving context.  
> 
> 
>   What to do? 
> 
> 
>   scratch/wow64stack contains a program that detects the bug.
>   I believe it is the basis of a workaround for the bug.   
> 
> 
>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>   not only functions that use exception handling, but also functions that call "extern" functions  
>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>   use that, if it isn't zero. 
>   Normally it will be zero.  
>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>   That is why it isn't a set/set-zero pattern (like in the test program).   
> 
> 
>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
> 
> 
>   Rejected counter proposal: 
>    Don't suspend/gc when running a syscall.  
>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
> 
> 
>  Possible augmentation: if running native, short circuit most of this 
> 
> 
>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
> 
> 
>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
> 
> 
>   Performance impact: hypothetically large but probably not noticable. 
> 
> 
>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>   We could further augment pragmas to discern them. 
> 
> 
>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
> 
> 
> 
>   Agreed? I'll make the compiler change? 
> 
> 
> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
> 
> 
> Eventually cooperative suspend will cause this to fall away as a problem..
> 
> 
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/df5802c0/attachment-0001.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:12:33 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:12:33 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
Message-ID: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:

> Is our garbage collector compacting?

Yes, it is mostly-copying.
Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).

> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?

It reads both registers and stack.

> Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.
Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.

> There are tradeoffs either way.

Indeed. Too many to enumerate here.

> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).

> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.

No updates to values on the stacks.

> If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.

> Should this all be parameters passed to the backend?

Nope.

> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.

Nope.

> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.


It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/12b01775/attachment-0001.html>

From jay.krell at cornell.edu  Sat Jan 25 05:04:59 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 25 Jan 2014 04:04:59 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
Message-ID: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>

I might yet have another solution here..there might be a way to detect the race and retry...later..

 - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:24:35 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified.  Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - JayFrom: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 



 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.  Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.  Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.  Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140125/b19a595a/attachment-0001.html>

From hendrik at topoi.pooq.com  Sat Jan 25 05:38:03 2014
From: hendrik at topoi.pooq.com (Hendrik Boom)
Date: Fri, 24 Jan 2014 23:38:03 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
Message-ID: <20140125043803.GA3423@topoi.pooq.com>

On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> I might yet have another solution here..there might be a way to 
> detect the race and retry...later..
> 
>  - Jay

I seem to remember a lot of comments in the trestle code tht seem to 
be intended as assertions to catch race conditions.
Is any of that stuff conceivably helpful here?

-- hendrik


From jay.krell at cornell.edu  Sun Jan 26 07:08:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <20140125043803.GA3423@topoi.pooq.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>,
	<20140125043803.GA3423@topoi.pooq.com>
Message-ID: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>

No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140126/e01d9d37/attachment-0001.html>

From jay.krell at cornell.edu  Mon Jan 27 08:23:26 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:23:26 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	, <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>, ,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>, ,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>, ,
	<20140125043803.GA3423@topoi.pooq.com>,
	<COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
Message-ID: <COL130-W141C96275EF6018B1D2D0E6A20@phx.gbl>


 > I might yet have another solution here
And it might only be for Windows 8 or newer.I tend to think we should go with the earlier proposal -- save context ourselves in thread locals when calling externals. Likely portable implementation is setjmp or getcontext, though we can maybe do better.This would be part of cooperative suspend anyway, which we want to move to.

 - Jay

From: jay.krell at cornell.edu
To: hendrik at topoi.pooq.com; m3devel at elegosoft.com
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?




No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/146c0737/attachment-0001.html>

From jay.krell at cornell.edu  Mon Jan 27 08:45:08 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:45:08 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
Message-ID: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>

In summary, shortest:

  We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).    We must be able to read the stack.    We need not write registers or stack (from garbage collector).  

Slight elaboration: Copying/compaction only occurs when references are only from globals and heap.  Therefore updates/writes only occur to globals and heap. 

 Is there any desire to do better/different?  To require the ability to update stack/registers?  What does Java typically to? 


I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.To do better/different would require more backend cooperation.We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Thanks, - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:12:33 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] garbage collector compacting and volatile?

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
Is our garbage collector compacting?

Yes, it is mostly-copying.Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?

It reads both registers and stack.
Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
There are tradeoffs either way.

Indeed. Too many to enumerate here.
If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.

No updates to values on the stacks.
If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.
Should this all be parameters passed to the backend?

Nope.
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.

Nope.
If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.

It is mostly-copying.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/9dd883b4/attachment-0001.html>

From hosking at cs.purdue.edu  Mon Jan 27 16:16:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 27 Jan 2014 10:16:57 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
Message-ID: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>



On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:

> In summary, shortest:
> 
> 
>   We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).  
>   We must be able to read the stack.  
>   We need not write registers or stack (from garbage collector).  

Correct.

> Slight elaboration:
>  Copying/compaction only occurs when references are only from globals and heap. 
>  Therefore updates/writes only occur to globals and heap. 

Correct.

>  Is there any desire to do better/different? 
>  To require the ability to update stack/registers? 

Without control of the register allocator, optimisations, and stack layout performed by the compiler (especially back-end) getting fully accurate stack maps is very difficult.

>  What does Java typically to? 

Java VMs usually control their own destiny, having full control of JIT and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.

> I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.

Correct.

> Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.

Correct.

> To do better/different would require more backend cooperation.
> We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Correct.

> 
> 
> Thanks,
>  - Jay
> 
> From: hosking at cs.purdue.edu
> Date: Fri, 24 Jan 2014 13:12:33 -0500
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
> 
> Is our garbage collector compacting?
> 
> Yes, it is mostly-copying.
> Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
> 
> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?
> 
> It reads both registers and stack.
> 
> Update references anywhere? Just the stack? Also context?
> 
> The collector first pins all pages referenced from the stacks, including registers.
> Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
> 
> There are tradeoffs either way.
> 
> Indeed. Too many to enumerate here.
> 
> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?
> 
> The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
> 
> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.
> 
> No updates to values on the stacks.
> 
> If we read and write registers in the collector, then nothing has to be volatile.
> 
> Read only.  No writes to registers.
> 
> Should this all be parameters passed to the backend?
> 
> Nope.
> 
> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.
> 
> Nope.
> 
> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.
> 
> It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/6f9b0cfd/attachment-0001.html>

From rodney_bates at lcwb.coop  Mon Jan 27 20:46:23 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 13:46:23 -0600
Subject: [M3devel] New compile failure
Message-ID: <52E6B78F.7000003@lcwb.coop>

The compiler in the head, with some recent changes, gives:

rodney at yellowstone:~/proj/m3/cm3-head/cm3/m3-sys/m3middle$ cm3

Fatal Error: configuration file didn't specify BUILD_DIR

This message is coming from cm3/src/Main.m3, which has recently
changed, but there are related changes to other source files
too.  Reverting Main.m3 to the previous version won't compile.

Putting an older executable for cm3 into the bin directory,
with recently updated and installed cm3.cfg and config works.


From rodney_bates at lcwb.coop  Mon Jan 27 21:49:43 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 14:49:43 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E13487.9090504@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
	<52E13487.9090504@lcwb.coop>
Message-ID: <52E6C667.6030708@lcwb.coop>

It should be fixed in the head, now.

On 01/23/2014 09:25 AM, Rodney M. Bates wrote:
>
>
> On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
>> Aha so declaring the type as <: ROOT will probably work as a workaround?
>>
>
> I'm not sure whether that would do it or not.  In my pared-down case, I had
> moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
> the failures go away.
>
> Until it is fixed, I think the best workaround is TYPECASE with a binding:
>
> TYPECASE x
> OF BDD.T(b) =>
>    Format(b), etc.
>
> This is known to be working now.  You've probably thought about this, but
> this way requires only a single runtime type test.  Either TYPECASE without
> the binding of b, or ISTYPE, will require an additional recheck of the same
> thing when the narrow, implicit or explicit, happens.  I doubt even a
> pretty good optimizing compiler could remove this.  For ISTYPE, it would have
> to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
> unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
> relation between CheckIsType and ScanTypecase.
>
>> Thank you for looking into this.
>>
>> "Rodney M. Bates" writes:
>>> Some more info:
>>>
>>> The front end is generating incorrect narrow code, only
>>> where the type T is an unrevealed <: REFANY.  When T and Root
>>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>>> and implicit narrow in an assignment.
>>>
>>> When opaque, for explicit and implicit narrow, it simply compares
>>> the typecode of x to a single typecode (no doubt for T) and demands
>>> they be equal.  In this case, the actual allocated type of x is Root,
>>> a proper subtype of T.
>>>
>>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>>
>>> The fact that it happens only when the types are opaque lends
>>> credibility to the fact that this bug has gone unnoticed at least
>>> as far back as PM3.
>>
>
>



From jay.krell at cornell.edu  Tue Jan 28 00:13:46 2014
From: jay.krell at cornell.edu (Jay)
Date: Mon, 27 Jan 2014 15:13:46 -0800
Subject: [M3devel] Fwd: Why does std::chrono now() uses slow syscall?
References: <CAH6eHdSjowHv+AM=T7nKgyoVuthbg6dAVbBvP0+UBxg0w=_jgg@mail.gmail.com>
Message-ID: <9FD55748-9752-4E40-9605-78F98AE62A5D@gmail.com>

Hints for how m3core/libm3 should get time efficiently...

 - Jay

Begin forwarded message:

> From: Jonathan Wakely <jwakely.gcc at gmail.com>
> Date: January 26, 2014, 2:33:36 AM PST
> To: Keith Erickson <kerickso at pppl.gov>
> Cc: gcc-help <gcc-help at gcc.gnu.org>
> Subject: Re: Why does std::chrono now() uses slow syscall?
> 
> On 26 January 2014 07:50, Keith Erickson wrote:
>> My questions, then, are....  why would I ever want to use the syscall
>> version?  It seems that configure prefers it, and will use it if it's
>> available.
> 
> It prefers to use clock_gettime and only uses the syscall if a
> configure test using clock_gettime(CLOCK_MONOTONIC, &tp) fails.
> 
>> But it's stupidly slow, even on the fasted server CPU that
>> AMD sells (Opteron 6386 SE at the time of this writing).  How do I not
>> use this slow method?  Do I have to compile gcc specially?  What are
>> the drawbacks to forcing configure to not set that #define?  Is there
>> an approved way to tell configure to use a fast time?
> 
> Prior to glibc 2.17 clock_gettime was in librt.so not libc.so, which
> is not used automatically by libstdc++. To tell configure to use it
> you need to build GCC with --enable-libstdcxx-time=rt, but that means
> that libstdc++.so will be linked to librt.so which depends on
> libpthread.so which means that the library always thinks it is part of
> a multithreaded application and so even single-threaded programs use
> atomic operations for internal reference-counting (e.g. in std::string
> and std::shared_ptr).  That's why we don't use clock_gettime unless
> explicitly configured to do so.
> 
> With glibc 2.17 or later clock_gettime will be used automatically,
> because the functions were moved out of librt.so.
> 
> On my Fedora 20 system I get
> 
> | #define HAVE_SYS_TIME_H 1
> | #define _GLIBCXX_USE_GETTIMEOFDAY 1
> | #define _GLIBCXX_USE_CLOCK_MONOTONIC 1
> | #define _GLIBCXX_USE_CLOCK_REALTIME 1
> | #define _GLIBCXX_USE_SCHED_YIELD 1
> | #define _GLIBCXX_USE_NANOSLEEP 1
> 
> which means it doesn't need the syscall version.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/d9369ca7/attachment-0001.html>

From mika at async.caltech.edu  Wed Jan 29 00:49:19 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 28 Jan 2014 15:49:19 -0800
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
Message-ID: <20140128234919.0E0C81A208F@async.async.caltech.edu>


Sounds like Jay doesn't need to buy a copy of your book :-)

Tony Hosking writes:
>
>--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=iso-8859-1
>
>
>
>On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>
>> In summary, shortest:
>>=20
>>=20
>>   We must be able to read registers (or save them to the stack or =
>elsewhere (known thread locals)). =20
>>   We must be able to read the stack. =20
>>   We need not write registers or stack (from garbage collector). =20
>
>Correct.
>
>> Slight elaboration:
>>  Copying/compaction only occurs when references are only from globals =
>and heap.=20
>>  Therefore updates/writes only occur to globals and heap.=20
>
>Correct.
>
>>  Is there any desire to do better/different?=20
>>  To require the ability to update stack/registers?=20
>
>Without control of the register allocator, optimisations, and stack =
>layout performed by the compiler (especially back-end) getting fully =
>accurate stack maps is very difficult.
>
>>  What does Java typically to?=20
>
>Java VMs usually control their own destiny, having full control of JIT =
>and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>
>> I guess there is the problem that we don't know for certain if values =
>in registers and stack are pointers, or just coincident integers, and so =
>updating them can be wrong.
>
>Correct.
>
>> Whereas in the globals and "reachable thereof" we have precise type =
>information for, so we can update those values safely.
>
>Correct.
>
>> To do better/different would require more backend cooperation.
>> We conservatively treat registers/stack as pointers if they look like =
>them, and therefore keep values alive, but not trust them so much as to =
>update them.
>
>Correct.


From jay.krell at cornell.edu  Wed Jan 29 01:00:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 29 Jan 2014 00:00:20 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>, ,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>,
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>,
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>,
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <COL130-W51823A67DAA7B4D7B7BDD3E6AC0@phx.gbl>

Good point.  I have it around somewhere... (I need that automatic resource management in real life..)
 
 - Jay

 
> To: hosking at cs.purdue.edu
> Date: Tue, 28 Jan 2014 15:49:19 -0800
> From: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
> >
> >--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
> >Content-Transfer-Encoding: quoted-printable
> >Content-Type: text/plain;
> >	charset=iso-8859-1
> >
> >
> >
> >On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
> >
> >> In summary, shortest:
> >>=20
> >>=20
> >>   We must be able to read registers (or save them to the stack or =
> >elsewhere (known thread locals)). =20
> >>   We must be able to read the stack. =20
> >>   We need not write registers or stack (from garbage collector). =20
> >
> >Correct.
> >
> >> Slight elaboration:
> >>  Copying/compaction only occurs when references are only from globals =
> >and heap.=20
> >>  Therefore updates/writes only occur to globals and heap.=20
> >
> >Correct.
> >
> >>  Is there any desire to do better/different?=20
> >>  To require the ability to update stack/registers?=20
> >
> >Without control of the register allocator, optimisations, and stack =
> >layout performed by the compiler (especially back-end) getting fully =
> >accurate stack maps is very difficult.
> >
> >>  What does Java typically to?=20
> >
> >Java VMs usually control their own destiny, having full control of JIT =
> >and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
> >
> >> I guess there is the problem that we don't know for certain if values =
> >in registers and stack are pointers, or just coincident integers, and so =
> >updating them can be wrong.
> >
> >Correct.
> >
> >> Whereas in the globals and "reachable thereof" we have precise type =
> >information for, so we can update those values safely.
> >
> >Correct.
> >
> >> To do better/different would require more backend cooperation.
> >> We conservatively treat registers/stack as pointers if they look like =
> >them, and therefore keep values alive, but not trust them so much as to =
> >update them.
> >
> >Correct.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140129/895ab0a2/attachment-0001.html>

From hosking at cs.purdue.edu  Wed Jan 29 01:07:15 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 28 Jan 2014 19:07:15 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <38455303-D2F4-4BDD-8E31-D7139D566436@cs.purdue.edu>

;-)

I should send him a copy!

On Jan 28, 2014, at 6:49 PM, mika at async.caltech.edu wrote:

> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
>> 
>> --Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=iso-8859-1
>> 
>> 
>> 
>> On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>> 
>>> In summary, shortest:
>>> =20
>>> =20
>>>  We must be able to read registers (or save them to the stack or =
>> elsewhere (known thread locals)). =20
>>>  We must be able to read the stack. =20
>>>  We need not write registers or stack (from garbage collector). =20
>> 
>> Correct.
>> 
>>> Slight elaboration:
>>> Copying/compaction only occurs when references are only from globals =
>> and heap.=20
>>> Therefore updates/writes only occur to globals and heap.=20
>> 
>> Correct.
>> 
>>> Is there any desire to do better/different?=20
>>> To require the ability to update stack/registers?=20
>> 
>> Without control of the register allocator, optimisations, and stack =
>> layout performed by the compiler (especially back-end) getting fully =
>> accurate stack maps is very difficult.
>> 
>>> What does Java typically to?=20
>> 
>> Java VMs usually control their own destiny, having full control of JIT =
>> and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>> 
>>> I guess there is the problem that we don't know for certain if values =
>> in registers and stack are pointers, or just coincident integers, and so =
>> updating them can be wrong.
>> 
>> Correct.
>> 
>>> Whereas in the globals and "reachable thereof" we have precise type =
>> information for, so we can update those values safely.
>> 
>> Correct.
>> 
>>> To do better/different would require more backend cooperation.
>>> We conservatively treat registers/stack as pointers if they look like =
>> them, and therefore keep values alive, but not trust them so much as to =
>> update them.
>> 
>> Correct.



From mika at async.caltech.edu  Wed Jan  1 11:48:36 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 01 Jan 2014 02:48:36 -0800
Subject: [M3devel] mklib not compiling
Message-ID: <20140101104836.E13C51A208E@async.async.caltech.edu>

Hi m3devel,

Trying to upgrade my compiler on AMD64_LINUX...

=== package /home/mika/cm3-cvs-anon/cm3/m3-sys/mklib ===
 +++ cm3 -build -DROOT='/home/mika/cm3-cvs-anon/cm3' $RARGS  && cm3 -ship $RARGS -DROOT='/home/mika/cm3-cvs-anon/cm3'  +++
--- building in AMD64_LINUX ---

ignoring ../src/m3overrides

new source -> compiling Main.m3
"../src/Main.m3", line 659: incompatible types (b)
"../src/Main.m3", line 660: incompatible types (b)
"../src/Main.m3", line 661: incompatible types (b)
"../src/Main.m3", line 662: incompatible types (b)
"../src/Main.m3", line 663: incompatible types (b)
"../src/Main.m3", line 664: incompatible types (b)
"../src/Main.m3", line 665: incompatible types (b)
7 errors encountered
compilation failed => not building program "mklib"
Fatal Error: package build failed

I'm at the head...

    Mika


From mika at async.caltech.edu  Thu Jan  2 17:34:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 08:34:10 -0800
Subject: [M3devel] building the CVS head---how???
Message-ID: <20140102163410.757DB1A2080@async.async.caltech.edu>


So here is the situation.

New machine, new problems.  

AMD64_LINUX.  I want to try the latest compiler, because there are issues with what I have.

What I have is the following:

Critical Mass Modula-3 version 5.8.6
  last updated: 2010-04-11
  compiled: 2010-07-12 20:10:34
  configuration: /home/mika/cm3.amd64-linux/bin/cm3.cfg
  host: AMD64_LINUX
  target: AMD64_LINUX

I think the boot archives on the CM3 web site are about the same age.

I try 

./boot1.py AMD64_LINUX

and I get

"../src/win32/WinSock.i3", line 662: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 665: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 688: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 691: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 694: unsupported language or calling convention (PASCAL)
"../src/win32/WinSock.i3", line 697: unsupported language or calling convention (PASCAL)
34 errors encountered
new exporters -> recompiling WinVer.i3
"../src/win32/WinVer.i3", line 158: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 168: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 180: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 190: unsupported language or calling convention (APIENTRY)
"../src/win32/WinVer.i3", line 203: unsupported language or calling convention (APIENTRY)

(etc)

all right, it's broken, something needs to be fixed.

WHAT IS NOT ALL RIGHT:

At this point my regular cm3.cfg is messed up.  If I had been silly enough to try to bootstrap
a new compiler (for a different machine) without first saving my old CM3 installation, I'd
be screwed.

But from reading the mailing list logs, I am not sure if it is even possible to bootstrap
the head from the older installation.  How am I supposed to get a new compiler working?
Build it on my Raspberry Pi (which has a working, recent CM3 on it)?  (After backing up
my old installation, of course.)

And what's this?  Bad python?

(119)truffles:~/cm3-cvs-anon/cm3/scripts/python>./boot1.py AMD64_LINUX
Traceback (most recent call last):
  File "./boot1.py", line 5, in <module>
    import pylib
  File "/big/home2/mika/2/cm3-cvs/cm3/scripts/python/pylib.py", line 587, in <module>
    for a in os.popen(CM3 + " -version 2>" + DevNull):
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
(120)truffles:~/cm3-cvs-anon/cm3/scripts/python>python --version
Python 2.7.3

     Mika
     Mika


From mika at async.caltech.edu  Fri Jan  3 00:15:37 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 15:15:37 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
Message-ID: <20140102231537.B953A1A2080@async.async.caltech.edu>


So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing

./boot1.py c AMD64_LINUX

I get:

new source -> compiling PklActionSeq.i3
new source -> compiling ConvertPacking.m3
"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)


***
*** runtime error:
***    <*ASSERT*> failed.
***    file "../src/M3C.m3", line 5890
***

Aborted
 *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***


   5887 PROCEDURE shift_right(self: T; type: IType) =
   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
   5889 BEGIN
   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
   5891     shift_left_or_right(self, type, "shift_right", ">>");
   5892 END shift_right;

...

Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...

     Mika


From mika at async.caltech.edu  Fri Jan  3 06:12:52 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:12:52 -0800
Subject: [M3devel] booting for AMD64_LINUX, continued...
In-Reply-To: <20140102174632.3169C41A@resin13.mta.everyone.net>
References: <20140102174632.3169C41A@resin13.mta.everyone.net>
Message-ID: <20140103051252.88FC61A2080@async.async.caltech.edu>

Well I managed to get it to work using the following roundabout procedure:

Since the only archives I could find on elegosoft for AMD64_LINUX were ancient,
I installed VirtualBox and FreeBSD/amd64 inside that.  FreeBSD/amd64 for some
reason has very recent archives (of the old "cminstall" kind).

Downloaded the FreeBSD archive, cminstalled it, and ran upgrade.py on the latest
sources.

Had to remove some code from ConvertPacking.m3 because Jay's C-backend can't compile
that for some reason.

ran boot1.py, which gave me a boot archive.  Note---if you've done this with boot1.py "c"
before, you have to delete all the derived dirs and start over, or you get a mess.

Moved the boot archive to the Linux machine.

Resurrected ConvertPacking.m3

Ran boot2.sh (or equiv) until I got everything built.

That worked.

So... some observations: 

.. I have a very recent build on ARM_LINUX, but wasn't able to cross-compile.  Word size
problem?

.. I now have three very recent builds: ARM_LINUX, AMD64_FREEBSD, and AMD64_LINUX.  What
precisely do I have to do to make boot archives out of these so other people might be spared
this painful process?  Very few archives on the elego site are up to date.  Some NT stuff and
some FreeBSD stuff.  Nothing else that I can see.

    Mika

"Rodney Bates" writes:
>
>
>-Rodney Bates
>
>--- mika at async.caltech.edu wrote:
>
>From: mika at async.caltech.edu
>To: m3devel at elegosoft.com
>Subject: [M3devel] booting for AMD64_LINUX, continued...
>Date: Thu, 02 Jan 2014 15:15:37 -0800
>
>
>So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing
>
>./boot1.py c AMD64_LINUX
>
>I get:
>
>new source -> compiling PklActionSeq.i3
>new source -> compiling ConvertPacking.m3
>"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21)
>"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21)
>
>
>***
>*** runtime error:
>***    <*ASSERT*> failed.
>***    file "../src/M3C.m3", line 5890
>***
>
>Aborted
> *** execution of [<function _BuildLocalFunction at 0x35e1f0>] failed ***
>
>
>   5887 PROCEDURE shift_right(self: T; type: IType) =
>   5888 (* s1.type := Word.Shift  (s1.type, -s0.type); pop *)
>   5889 BEGIN
>   5890     <* ASSERT cgtypeIsUnsignedInt[type] *>
>   5891     shift_left_or_right(self, type, "shift_right", ">>");
>   5892 END shift_right;
>
>...
>
>Does anyone have a relatively recent working AMD64_LINUX installation?  I'd love a copy to try to boot from...
>
>I have one, but am out of the country right now and have no access to it.  
>I am waiting as I write to board the first airplane home, so will try to
>tar one up for you when I get there. 
>
>
>     Mika
>


From mika at async.caltech.edu  Fri Jan  3 06:57:18 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Thu, 02 Jan 2014 21:57:18 -0800
Subject: [M3devel] pthreads still broken
Message-ID: <20140103055718.B12821A2080@async.async.caltech.edu>

On my new AMD64_LINUX build...

The thread tester still crashes.

First, it prints a couple of extraneous diagnostics.  These look like
they come from the threading code?

Then it segfaults.

(141)truffles:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX>gdb threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest...done.
(gdb) run 
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x2aaaab6f8700 (LWP 23996)]
[New Thread 0x2aaaab8f9700 (LWP 23997)]
[New Thread 0x2aaaabafa700 (LWP 23998)]
done
Creating fork threads...[New Thread 0x2aaaabcfb700 (LWP 23999)]
[New Thread 0x2aaaabefc700 (LWP 24000)]
[New Thread 0x2aaaac0fd700 (LWP 24001)]
done
Creating alloc threads...[New Thread 0x2aaaac2fe700 (LWP 24002)]
[New Thread 0x2aaaac4ff700 (LWP 24003)]
[New Thread 0x2aaaac700700 (LWP 24004)]
done
Creating lock threads...[New Thread 0x2aaaac901700 (LWP 24005)]
[New Thread 0x2aaaacb02700 (LWP 24006)]
[New Thread 0x2aaaacd03700 (LWP 24007)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x2aaaac901700 (LWP 24005)]
0x00002aaaaaf5b64b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x2aaaab6f8700 (LWP 23996)]
0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
44          IF (rd.buff = NIL) THEN
(gdb) where
#0  0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=<error reading variable>, M3_EQj1Bs_h=<error reading variable>) at ../src/rw/FileRd.m3:44
#1  0x0000000000416d3a in FileRd__Open (M3_Bd56fi_p=<error reading variable>) at ../src/rw/FileRd.m3:16
#2  0x0000000000405424 in Main__RApply (M3_AP7a1g_cl=<error reading variable>) at ../src/Main.m3:182
#3  0x000000000044c23b in ThreadPThread__RunThread (M3_DMxDjQ_me=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:449
#4  0x000000000044bef8 in ThreadPThread__ThreadBase (M3_AJWxb1_param=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#5  0x00002aaaaaf56b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00002aaaab246a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x0000000000000000 in ?? ()
(gdb) 

The reason I even ran it was because I found that running the parallel compiler backend was unstable. 
  
Do user threads work on AMD64_LINUX?

     Mika


From mika at async.caltech.edu  Fri Jan  3 17:13:43 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 08:13:43 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
Message-ID: <20140103161343.F25291A2080@async.async.caltech.edu>

(78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LINUX/threadtest
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest...done.
(gdb) handle SIGILL 
Signal        Stop      Print   Pass to program Description
SIGILL        Yes       Yes     Yes             Illegal instruction
(gdb) handle SIGILL  nostop noprint
Signal        Stop      Print   Pass to program Description
SIGILL        No        No      Yes             Illegal instruction
(gdb) run
Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Writing file...done
Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
[New Thread 0x40bf4450 (LWP 2942)]
[New Thread 0x40df4450 (LWP 2943)]
done
Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
[New Thread 0x411f4450 (LWP 2945)]
[New Thread 0x413f4450 (LWP 2946)]
done
Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
[New Thread 0x417f4450 (LWP 2948)]
[New Thread 0x419f4450 (LWP 2949)]
done
Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
[New Thread 0x41df4450 (LWP 2951)]
[New Thread 0x41ff4450 (LWP 2952)]
done
running...printing oldest/median age/newest

Program received signal SIG64, Real-time event 64.
[Switching to Thread 0x41ff4450 (LWP 2952)]
0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0
(gdb) handle SIG64 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG64         No        No      Yes             Real-time event 64
(gdb) cont
Continuing.
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "../src/runtime/common/RTCollector.m3", line 418
***


Program received signal SIGABRT, Aborted.
[Switching to Thread 0x40ff4450 (LWP 2944)]
0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) where
#0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
#2  0x00000000 in ?? ()
(gdb) 

It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
ARM_LINUX but would be surprised if it didn't work.

Yes the pthreads seem "more reliable" than they used to be, but I would
never use them in any sort of mission-critical application.  I use
Modula-3 programs for (among other things) financial transactions and
real-time motor controllers...

    Mika


From mika at async.caltech.edu  Fri Jan  3 18:19:32 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 03 Jan 2014 09:19:32 -0800
Subject: [M3devel] AMD64_LINUX update...
Message-ID: <20140103171932.F28D61A2080@async.async.caltech.edu>


Hi again m3devel,

With the head and user threads, after bootstrapping via AMD64_FREEBSD,
I'm getting everything (apparently) working EXCEPT one thing:

stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
into an infinite loop, looks like it's happening on the stack because I
run out of memory rather quickly, on certain types local to my own system.

Has anyone seen this before?  I've never seen it on any 32-bit system
but I have a vague memory of seeing it on 64-bit systems.

Will debug over the weekend...

I'd be grateful for any input from the list on the pthreads issues, though.

     Mika


From mika at async.caltech.edu  Sat Jan  4 19:55:11 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 10:55:11 -0800
Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX
Message-ID: <20140104185511.9000A1A2080@async.async.caltech.edu>


I have this program that generates Modula-3 code from XML.

It uses Wr.PutText a lot, on small strings.

It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
Core i7 4960X.

(176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w

(29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w

I believe the issue is that Text.m3 is making a ton of system calls!

Here's a typical traceback:

(gdb) where
#0  0x00002aaaaaacb770 in ?? ()
#1  0x00002aaaaaacba1b in gettimeofday ()
#2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
#3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
#4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
#5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
#6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>, 
    M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
#7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
#8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
#9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>, 
    M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
#10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
#11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
#12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>, 
    M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
#13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
#14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>, 
    M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>, 
    M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>, 
    M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
#15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
#16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
(gdb) 

I believe all the TextStats operations are killing the performance.  Can we remove them?

Here is SetChars:

PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
  BEGIN
    TextStats.NoteGround (TextStats.Op.SetChars);
    TextStats.NoteGround (TextStats.Op.get_chars);
    t.get_chars (a, start);
    TextStats.NoteFinished (TextStats.Op.get_chars);
    TextStats.NoteFinished (TextStats.Op.SetChars);
  END SetChars;

I'm not sure how best to do this.  Removing them without killing functionality might not be
easy.  Looks like a case for conditional compilation.

Maybe generate the .m3s from some other source, with and without the TextStats, depending on
a setting in m3makefile?

Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.  

     Mika



From dragisha at m3w.org  Sat Jan  4 22:04:22 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:04:22 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140103171932.F28D61A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
Message-ID: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>

Hi Mika,

I have this for LINUXLIBC6, and I am using it for remake of HEAD: cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz

Can you build and share cm3-bin-core for AMD64_LINUX?

TIA,
dd

On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:

> 
> Hi again m3devel,
> 
> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
> I'm getting everything (apparently) working EXCEPT one thing:
> 
> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
> into an infinite loop, looks like it's happening on the stack because I
> run out of memory rather quickly, on certain types local to my own system.
> 
> Has anyone seen this before?  I've never seen it on any 32-bit system
> but I have a vague memory of seeing it on 64-bit systems.
> 
> Will debug over the weekend...
> 
> I'd be grateful for any input from the list on the pthreads issues, though.
> 
>     Mika

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/b89b19fd/attachment-0002.sig>

From mika at async.caltech.edu  Sat Jan  4 22:12:00 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:12:00 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
Message-ID: <20140104211200.A80571A2080@async.async.caltech.edu>

Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?

I can do this for both AMD64_LINUX (user threads)
and ARM_LINUX (pthreads)

I might still have AMD64_LINUX (pthreads)

Also have AMD64_FREEBSD (pthreads) from the head.

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=us-ascii
>
>Hi Mika,
>
>I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>
>Can you build and share cm3-bin-core for AMD64_LINUX?
>
>TIA,
>dd
>
>On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>
>>=20
>> Hi again m3devel,
>>=20
>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>> I'm getting everything (apparently) working EXCEPT one thing:
>>=20
>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>> into an infinite loop, looks like it's happening on the stack because =
>I
>> run out of memory rather quickly, on certain types local to my own =
>system.
>>=20
>> Has anyone seen this before?  I've never seen it on any 32-bit system
>> but I have a vague memory of seeing it on 64-bit systems.
>>=20
>> Will debug over the weekend...
>>=20
>> I'd be grateful for any input from the list on the pthreads issues, =
>though.
>>=20
>>     Mika
>
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/html;
>	charset=us-ascii
>
><html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; font-family: Candara; =
>border-spacing: 0px;"><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><span class=3D"Apple-style-span" =
>style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>Helvetica; font-style: normal; font-variant: normal; font-weight: =
>normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>white-space: normal; widows: 2; word-spacing: 0px; =
>-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>space; -webkit-line-break: after-white-space; ">dd</div><div =
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>-webkit-line-break: after-white-space; =
>"><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>at 18:19, <a =
>href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>infinite loop, looks like it's happening on the stack because I<br>run =
>out of memory rather quickly, on certain types local to my own =
>system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>for any input from the list on the pthreads issues, though.<br><br> =
>&nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>=
>
>--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>Content-Transfer-Encoding: 7bit
>Content-Disposition: attachment;
>	filename=signature.asc
>Content-Type: application/pgp-signature;
>	name=signature.asc
>Content-Description: Message signed with OpenPGP using GPGMail
>
>-----BEGIN PGP SIGNATURE-----
>Comment: GPGTools - http://gpgtools.org
>
>iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>=sMKJ
>-----END PGP SIGNATURE-----
>
>--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--


From mika at async.caltech.edu  Sat Jan  4 22:15:51 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:15:51 -0800
Subject: [M3devel] thread tester results on ARM_LINUX
In-Reply-To: <EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
References: <20140103161343.F25291A2080@async.async.caltech.edu>
	<EBA70433-269F-4593-BABF-E1B6DFD7CE0A@purdue.edu>
Message-ID: <20140104211551.54F481A2080@async.async.caltech.edu>

Antony Hosking writes:
>I wonder of the redzone size for AMD 64 stack frames is not set properly.=20=
>

Tony,

Are you talking about the difference between the ARM and AMD64 results?

The AMD64 is also a machine with 12 logical (HT) processors whereas the
ARM is a single processor.  That could make a difference in how it fails.

By the way, would you know:  The following output on the amd64:

Continuing.
....pthread_mutex_destroy:EBUSY
...pthread_mutex_destroy:EBUSY

Is that an assertion failing in the pthreads library?  My program isn't
printing this stuff: it's coming from ThreadPThread or some such code.

     Mika

>
>Sent from my iPhone
>
>> On Jan 3, 2014, at 10:13 AM, mika at async.caltech.edu wrote:
>>=20
>> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LIN=
>UX/threadtest
>> GNU gdb (GDB) 7.4.1-debian
>> Copyright (C) 2012 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.ht=
>ml>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"=
>
>> and "show warranty" for details.
>> This GDB was configured as "arm-linux-gnueabihf".
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>...
>> Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/th=
>read/ARM_LINUX/threadtest...done.
>> (gdb) handle SIGILL=20
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        Yes       Yes     Yes             Illegal instruction
>> (gdb) handle SIGILL  nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIGILL        No        No      Yes             Illegal instruction
>> (gdb) run
>> Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/threa=
>d/ARM_LINUX/threadtest=20
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.=
>1".
>> Writing file...done
>> Creating read threads...[New Thread 0x409f4450 (LWP 2941)]
>> [New Thread 0x40bf4450 (LWP 2942)]
>> [New Thread 0x40df4450 (LWP 2943)]
>> done
>> Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)]
>> [New Thread 0x411f4450 (LWP 2945)]
>> [New Thread 0x413f4450 (LWP 2946)]
>> done
>> Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)]
>> [New Thread 0x417f4450 (LWP 2948)]
>> [New Thread 0x419f4450 (LWP 2949)]
>> done
>> Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)]
>> [New Thread 0x41df4450 (LWP 2951)]
>> [New Thread 0x41ff4450 (LWP 2952)]
>> done
>> running...printing oldest/median age/newest
>>=20
>> Program received signal SIG64, Real-time event 64.
>> [Switching to Thread 0x41ff4450 (LWP 2952)]
>> 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnu=
>eabihf/libpthread.so.0
>> (gdb) handle SIG64 nostop noprint
>> Signal        Stop      Print   Pass to program Description
>> SIG64         No        No      Yes             Real-time event 64
>> (gdb) cont
>> Continuing.
>> .
>>=20
>> ***
>> *** runtime error:
>> ***    An array subscript was out of range.
>> ***    file "../src/runtime/common/RTCollector.m3", line 418
>> ***
>>=20
>>=20
>> Program received signal SIGABRT, Aborted.
>> [Switching to Thread 0x40ff4450 (LWP 2944)]
>> 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> (gdb) where
>> #0  0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6
>> #1  0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6
>> #2  0x00000000 in ?? ()
>> (gdb)=20
>>=20
>> It runs fine with user threads on AMD64_LINUX now.  Haven't tried that on
>> ARM_LINUX but would be surprised if it didn't work.
>>=20
>> Yes the pthreads seem "more reliable" than they used to be, but I would
>> never use them in any sort of mission-critical application.  I use
>> Modula-3 programs for (among other things) financial transactions and
>> real-time motor controllers...
>>=20
>>    Mika


From dragisha at m3w.org  Sat Jan  4 22:52:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 22:52:32 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104211200.A80571A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
Message-ID: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>

-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*

there they are. I don?t do this too often, but both do-cm3-core and do-cm3-min will do for what I need.

TIA,
dd

On 04 Jan 2014, at 22:12, mika at async.caltech.edu wrote:

> Sure... what do I do?  Is there a script?  Or do I tar up a selected set of directories?
> 
> I can do this for both AMD64_LINUX (user threads)
> and ARM_LINUX (pthreads)
> 
> I might still have AMD64_LINUX (pthreads)
> 
> Also have AMD64_FREEBSD (pthreads) from the head.
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657"
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=us-ascii
>> 
>> Hi Mika,
>> 
>> I have this for LINUXLIBC6, and I am using it for remake of HEAD: =
>> cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz
>> 
>> Can you build and share cm3-bin-core for AMD64_LINUX?
>> 
>> TIA,
>> dd
>> 
>> On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> Hi again m3devel,
>>> =20
>>> With the head and user threads, after bootstrapping via AMD64_FREEBSD,
>>> I'm getting everything (apparently) working EXCEPT one thing:
>>> =20
>>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes
>>> into an infinite loop, looks like it's happening on the stack because =
>> I
>>> run out of memory rather quickly, on certain types local to my own =
>> system.
>>> =20
>>> Has anyone seen this before?  I've never seen it on any 32-bit system
>>> but I have a vague memory of seeing it on 64-bit systems.
>>> =20
>>> Will debug over the weekend...
>>> =20
>>> I'd be grateful for any input from the list on the pthreads issues, =
>> though.
>>> =20
>>>    Mika
>> 
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/html;
>> 	charset=us-ascii
>> 
>> <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
>> charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi =
>> Mika,<div><br></div><div>I have this for LINUXLIBC6, and I am using it =
>> for remake of HEAD:&nbsp;<span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2=
>> 8.tgz</span></div><div><span style=3D"background-color: rgb(0, 0, 0); =
>> color: rgb(255, 59, 29); font-family: Monaco; font-size: =
>> 13px;"><br></span></div><div><span style=3D"font-family: Helvetica; =
>> orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and =
>> share cm3-bin-core for AMD64_LINUX?</span></div><div><div =
>> apple-content-edited=3D"true"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; font-family: Candara; =
>> border-spacing: 0px;"><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><span class=3D"Apple-style-span" =
>> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: =
>> Helvetica; font-style: normal; font-variant: normal; font-weight: =
>> normal; letter-spacing: normal; line-height: normal; orphans: 2; =
>> text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
>> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
>> auto; -webkit-text-stroke-width: 0px;  "><div style=3D"word-wrap: =
>> break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
>> after-white-space; "><br></div><div style=3D"word-wrap: break-word; =
>> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
>> ">TIA,</div><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
>> space; -webkit-line-break: after-white-space; ">dd</div><div =
>> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
>> -webkit-line-break: after-white-space; =
>> "><br></div></span></div></span></span></div><div><div>On 03 Jan 2014, =
>> at 18:19, <a =
>> href=3D"mailto:mika at async.caltech.edu">mika at async.caltech.edu</a> =
>> wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
>> type=3D"cite"><br>Hi again m3devel,<br><br>With the head and user =
>> threads, after bootstrapping via AMD64_FREEBSD,<br>I'm getting =
>> everything (apparently) working EXCEPT one thing:<br><br>stubgen (and my =
>> derivative of it, the Scheme-stubgen "sstubgen") goes<br>into an =
>> infinite loop, looks like it's happening on the stack because I<br>run =
>> out of memory rather quickly, on certain types local to my own =
>> system.<br><br>Has anyone seen this before? &nbsp;I've never seen it on =
>> any 32-bit system<br>but I have a vague memory of seeing it on 64-bit =
>> systems.<br><br>Will debug over the weekend...<br><br>I'd be grateful =
>> for any input from the list on the pthreads issues, though.<br><br> =
>> &nbsp;&nbsp;&nbsp;&nbsp;Mika<br></blockquote></div><br></div></body></html=
>>> =
>> 
>> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657--
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421
>> Content-Transfer-Encoding: 7bit
>> Content-Disposition: attachment;
>> 	filename=signature.asc
>> Content-Type: application/pgp-signature;
>> 	name=signature.asc
>> Content-Description: Message signed with OpenPGP using GPGMail
>> 
>> -----BEGIN PGP SIGNATURE-----
>> Comment: GPGTools - http://gpgtools.org
>> 
>> iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+
>> taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM
>> zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7
>> 0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2
>> hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn
>> 1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg=
>> =sMKJ
>> -----END PGP SIGNATURE-----
>> 
>> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/de3b9c2d/attachment-0002.sig>

From mika at async.caltech.edu  Sat Jan  4 22:58:53 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 13:58:53 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
Message-ID: <20140104215853.09F0D1A2080@async.async.caltech.edu>

Hi dd,

I already have everything built.  

What I'm asking is how I go from what I have to a tar file.  What is
included?  Is there a script for that?

I can just give you the full binary directory or the full cm3 system with
sources, but I suspect there is a reduced set you would prefer?

BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
scripts in python work fine.  If you already have cm3 from the head
it ought to be possible to cross-build.  But I did have problems going from
32 to 64 bits...

     Mika

=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>
>--Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>Content-Type: multipart/alternative;
>	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>
>
>--Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=windows-1252
>
>-rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>-rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>-rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>-rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>-rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>-rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>-rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>-rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>-rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>
>there they are. I don=92t do this too often, but both do-cm3-core and =
>do-cm3-min will do for what I need.
>
>TIA,
>dd


From dragisha at m3w.org  Sat Jan  4 23:10:04 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:10:04 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104215853.09F0D1A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
Message-ID: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>

do-cm3-bin will build a tar.gz, I think. And it is not too big.

I did not try boot*.py as this box I would like to build for is remote and all? I have src.rpm and I only need this cm3-min? to make it work and build itself on remote box.

TIA,
dd

On 04 Jan 2014, at 22:58, mika at async.caltech.edu wrote:

> Hi dd,
> 
> I already have everything built.  
> 
> What I'm asking is how I go from what I have to a tar file.  What is
> included?  Is there a script for that?
> 
> I can just give you the full binary directory or the full cm3 system with
> sources, but I suspect there is a reduced set you would prefer?
> 
> BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh
> scripts in python work fine.  If you already have cm3 from the head
> it ought to be possible to cross-build.  But I did have problems going from
> 32 to 64 bits...
> 
>     Mika
> 
> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
>> 
>> --Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304
>> Content-Type: multipart/alternative;
>> 	boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12"
>> 
>> 
>> --Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=windows-1252
>> 
>> -rwxrwxr-x. 1 root  969 Feb  1  2010 scripts/do-cm3-all.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-base.sh*
>> -rwxrwxr-x. 1 root  994 Feb  1  2010 scripts/do-cm3-caltech-parser.sh*
>> -rwxrwxr-x. 1 root  998 Feb  1  2010 scripts/do-cm3-coll.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-comm.sh*
>> -rwxrwxr-x. 1 root  985 Feb  1  2010 scripts/do-cm3-core.sh*
>> -rwxrwxr-x. 1 root 1072 Feb  1  2010 scripts/do-cm3-front.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-gui.sh*
>> -rwxrwxr-x. 1 root  926 Jun  7  2009 scripts/do-cm3-m3gdb.sh*
>> -rwxrwxr-x. 1 root  984 Feb  1  2010 scripts/do-cm3-min.sh*
>> -rwxrwxr-x. 1 root  969 Nov 11  2010 scripts/do-cm3-std.sh*
>> -rwxrwxr-x. 1 root  938 Feb  6  2013 scripts/do-pkg.sh*
>> 
>> there they are. I don=92t do this too often, but both do-cm3-core and =
>> do-cm3-min will do for what I need.
>> 
>> TIA,
>> dd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/c46b6e8d/attachment-0002.sig>

From mika at async.caltech.edu  Sat Jan  4 23:23:21 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:23:21 -0800
Subject: [M3devel] can't write to repository
Message-ID: <20140104222321.B53051A2080@async.async.caltech.edu>

Hi m3devel,

I can't seem to write to the CVS repository.  I get:

(62)rover:~/cm3-writable/cm3/scripts>cvs up -d .
mika at birch.elegosoft.com's password: 

Has my account been removed?  I don't think I've changed keys.

In any case, could someone please commit the following change:

~/cm3-cvs-anon/cm3/scripts>cvs diff sysinfo.sh
Index: sysinfo.sh
===================================================================
RCS file: /usr/cvs/cm3/scripts/sysinfo.sh,v
retrieving revision 1.109
diff -r1.109 sysinfo.sh
273a274
>         arm*)    CM3_TARGET=ARM_LINUX;;

    Mika


From mika at async.caltech.edu  Sat Jan  4 23:48:13 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:48:13 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
Message-ID: <20140104224813.34D171A2080@async.async.caltech.edu>


I think the script you meant was make-bin-dist-min.sh

The results are at:

http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz

Please upload this to the CM3 website if it looks OK to you and you know how to upload it.

It's set up to use user threads.

    Mika



From dragisha at m3w.org  Sat Jan  4 23:57:55 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 4 Jan 2014 23:57:55 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104224813.34D171A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
Message-ID: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>

I am using it to bootstrap my rpm, did you try copying to: /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

I will make my rpm tomorrow, and try uploading all of it then.

Thanks,
dd



On 04 Jan 2014, at 23:48, mika at async.caltech.edu wrote:

> 
> I think the script you meant was make-bin-dist-min.sh
> 
> The results are at:
> 
> http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz
> 
> Please upload this to the CM3 website if it looks OK to you and you know how to upload it.
> 
> It's set up to use user threads.
> 
>    Mika
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140104/29728119/attachment-0002.sig>

From mika at async.caltech.edu  Sat Jan  4 23:59:24 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 14:59:24 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
Message-ID: <20140104225924.858A21A2080@async.async.caltech.edu>

>I am using it to bootstrap my rpm, did you try copying to: =
>/var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?

Per my email of a few minutes ago, I am unable to get birch to accept
my login credentials...


From dragisha at m3w.org  Sun Jan  5 01:44:17 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 01:44:17 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140104225924.858A21A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
Message-ID: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>

Copied, and did ?a sensible thing?, ran update script? It did not create as complete index.html as it was before. I will look into it tomorrow, it is too late for me now to believe my further fixing would not break more things :).


On 04 Jan 2014, at 23:59, mika at async.caltech.edu wrote:

>> I am using it to bootstrap my rpm, did you try copying to: =
>> /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch?
> 
> Per my email of a few minutes ago, I am unable to get birch to accept
> my login credentials...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/841337b6/attachment-0002.sig>

From mika at async.caltech.edu  Sun Jan  5 03:50:44 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 04 Jan 2014 18:50:44 -0800
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
Message-ID: <20140105025044.9FC551A2080@async.async.caltech.edu>


Another favor?

Could you upload to CM3 the file at

http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz 

If there is a "notes" section, you can note that the distribution works on 

1.  Raspberry Pi running Raspbian Wheezy
Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l

2.  BeagleBone Black running (BeagleBoard)Ubuntu 13.10 (GNU/Linux 3.8.13-bone28 armv7l)
Linux arm 3.8.13-bone28 #1 SMP Fri Sep 13 01:11:14 UTC 2013 armv7l armv7l armv7l GNU/Linux

(Anyone reading this mail who wants a copy can of course also use the link to rover above.)

     Mika


From dragisha at m3w.org  Sun Jan  5 21:15:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 5 Jan 2014 21:15:23 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <20140105025044.9FC551A2080@async.async.caltech.edu>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
Message-ID: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>

Also on http://opencm3.net/snaps/


On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:

> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140105/41d9d038/attachment-0002.sig>

From dragisha at m3w.org  Mon Jan  6 06:02:36 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 6 Jan 2014 06:02:36 +0100
Subject: [M3devel] AMD64_LINUX update...
In-Reply-To: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
References: <20140103171932.F28D61A2080@async.async.caltech.edu>
	<CC5F53C4-7950-4566-94E0-3389D9150F7E@m3w.org>
	<20140104211200.A80571A2080@async.async.caltech.edu>
	<FE0060E4-68FC-4862-8BFB-40103270CEE4@m3w.org>
	<20140104215853.09F0D1A2080@async.async.caltech.edu>
	<7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org>
	<20140104224813.34D171A2080@async.async.caltech.edu>
	<833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org>
	<20140104225924.858A21A2080@async.async.caltech.edu>
	<9CF45913-50ED-45F4-9774-0D0152016105@m3w.org>
	<20140105025044.9FC551A2080@async.async.caltech.edu>
	<6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org>
Message-ID: <9C223752-0979-4BC0-84C7-C8F83B9BF01B@m3w.org>

There is hourly run script for user m3 which updates index file in this folder. It keeps deleteng your AMD_LINUX snapshot, apparently after indexing it.

ARM_LINUX one is indexed and left in peace. I will look into this issue in few days.

dd

On 05 Jan 2014, at 21:15, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Also on http://opencm3.net/snaps/
> 
> 
> On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote:
> 
>> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140106/b87c8608/attachment-0002.sig>

From rodney_bates at lcwb.coop  Mon Jan  6 20:41:53 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 06 Jan 2014 13:41:53 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CB0701.9030307@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.
>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.

Try the fix I just checked in to TextClass.m3.  It was already set up with such a boolean flag
(TextClass.CollectStats, set FALSE), but in one of the instrumentation procedures, two system
calls were outside the test thereof.  Hopefully, this will give a good bit of relief for now,
with a simple fix.

>
>       Mika
>
>



From rodney_bates at lcwb.coop  Thu Jan  9 22:01:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 09 Jan 2014 15:01:55 -0600
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>
Message-ID: <52CF0E43.2030701@lcwb.coop>



On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> I have this program that generates Modula-3 code from XML.
>
> It uses Wr.PutText a lot, on small strings.
>
> It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> Core i7 4960X.
>
> (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
>
> (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
>
> I believe the issue is that Text.m3 is making a ton of system calls!
>
> Here's a typical traceback:
>
> (gdb) where
> #0  0x00002aaaaaacb770 in ?? ()
> #1  0x00002aaaaaacba1b in gettimeofday ()
> #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
>      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
>      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
>      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
>      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
>      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
>      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> (gdb)
>
> I believe all the TextStats operations are killing the performance.  Can we remove them?
>
> Here is SetChars:
>
> PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
>    BEGIN
>      TextStats.NoteGround (TextStats.Op.SetChars);
>      TextStats.NoteGround (TextStats.Op.get_chars);
>      t.get_chars (a, start);
>      TextStats.NoteFinished (TextStats.Op.get_chars);
>      TextStats.NoteFinished (TextStats.Op.SetChars);
>    END SetChars;
>
> I'm not sure how best to do this.  Removing them without killing functionality might not be
> easy.  Looks like a case for conditional compilation.

Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
only needed for performance study of the algorithms, which likely won't be done often.

>
> Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> a setting in m3makefile?
>
> Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
>
>       Mika
>
>



From jay.krell at cornell.edu  Fri Jan 10 09:22:25 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 10 Jan 2014 08:22:25 +0000
Subject: [M3devel] performance problems in CM3 Text operations on
 AMD64_LINUX
In-Reply-To: <52CF0E43.2030701@lcwb.coop>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
Message-ID: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>

Ok either, way, but here is another point.

In some of our code we do:

CONST DEBUG = FALSE;
IF FALSE THEN   xxEND;


It will tend to be optimized.Furthermore, syscalls -- even if not optimized,IF FALSE is going to be way faster than a syscall.

Also, gettimeofday shouldn't necessarily be a syscall.

But/and there are/should-be faster ways of getting time for benchmarking.

On Windows, GetTickCount and GetTickCount64 just read a value out of memory.Even GetSystemTimeAsFileTime is just a read of a 64bit value from memory.(The memory is read only, at the same address in all processes, and updated bythe kernel.)x86 has rdtsc, a 64bit hardware cycle counter readable in one instruction.



We should consider researching analogs to rdtsc across mips/powerpc/sparc/arm andprovide something in m3core. I think.Add it to the backend interface likely, for inlining.or maybe it is worth a function call, then just use #ifdefed C.


Maybe Posix provides something that tends to be like this?

Mika, how did you find this?Control-c every so often and look at callstack?Or a lot of stepping?

 - Jay

: Thu, 9 Jan 2014 15:01:55 -0600
> From: rodney_bates at lcwb.coop
> To: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] performance problems in CM3 Text operations on	AMD64_LINUX
> 
> 
> 
> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote:
> > I have this program that generates Modula-3 code from XML.
> >
> > It uses Wr.PutText a lot, on small strings.
> >
> > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium
> > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz
> > Core i7 4960X.
> >
> > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 0.419u 0.089s 0:00.70 70.0%     471+27608k 0+546io 0pf+0w
> >
> > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml
> > 1.952u 0.232s 0:05.40 40.3%     0+0k 0+11504io 0pf+0w
> >
> > I believe the issue is that Text.m3 is making a ton of system calls!
> >
> > Here's a typical traceback:
> >
> > (gdb) where
> > #0  0x00002aaaaaacb770 in ?? ()
> > #1  0x00002aaaaaacba1b in gettimeofday ()
> > #2  0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37
> > #3  0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50
> > #4  0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14
> > #5  0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=<error reading variable>) at ../src/text/TextStats.m3:64
> > #6  0x00000000004921de in Text__SetChars (M3_CKMnXU_a=<error reading variable>, M3_Bd56fi_t=<error reading variable>,
> >      M3_Cwb5VA_start=<error reading variable>) at ../src/text/Text.m3:512
> > #7  0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:100
> > #8  0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=<error reading variable>, M3_Bd56fi_t=<error reading variable>) at ../src/rw/Wr.m3:89
> > #9  0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=<error reading variable>, M3_BxxOH1_wr=<error reading variable>,
> >      M3_AicXUJ_quoted=<error reading variable>) at ../src/M3Syntax.m3:129
> > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:173
> > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=<error reading variable>, M3_ACGgGY_env=<error reading variable>) at ../src/M3Syntax.m3:295
> > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=<error reading variable>, M3_Bd56fi_dirPath=<error reading variable>,
> >      M3_AicXUJ_updateMakefile=<error reading variable>) at ../src/M3File.m3:78
> > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526
> > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=<error reading variable>, M3_Bd56fi_versionString=<error reading variable>,
> >      M3_Bd56fi_beginStringV=<error reading variable>, M3_D9m5ya_fieldTbl=<error reading variable>, M3_Bd56fi_Dir=<error reading variable>,
> >      M3_Bd56fi_SrcDir=<error reading variable>, M3_AZx9O5_binaryFields=<error reading variable>, M3_CebKnt_specialFields=<error reading variable>,
> >      M3_CT76zs_checkedTypes=<error reading variable>) at ../AMD64_LINUX/Fields.m3:538
> > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../AMD64_LINUX/Main.m3:431
> > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
> > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
> > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
> > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22
> > (gdb)
> >
> > I believe all the TextStats operations are killing the performance.  Can we remove them?
> >
> > Here is SetChars:
> >
> > PROCEDURE SetChars (VAR a: ARRAY OF CHAR;  t: T;  start: CARDINAL) =
> >    BEGIN
> >      TextStats.NoteGround (TextStats.Op.SetChars);
> >      TextStats.NoteGround (TextStats.Op.get_chars);
> >      t.get_chars (a, start);
> >      TextStats.NoteFinished (TextStats.Op.get_chars);
> >      TextStats.NoteFinished (TextStats.Op.SetChars);
> >    END SetChars;
> >
> > I'm not sure how best to do this.  Removing them without killing functionality might not be
> > easy.  Looks like a case for conditional compilation.
> 
> Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated
> by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly.  These are
> only needed for performance study of the algorithms, which likely won't be done often.
> 
> >
> > Maybe generate the .m3s from some other source, with and without the TextStats, depending on
> > a setting in m3makefile?
> >
> > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called.
> >
> >       Mika
> >
> >
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140110/fe64b4e8/attachment-0002.html>

From mika at async.caltech.edu  Fri Jan 10 16:27:10 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Fri, 10 Jan 2014 07:27:10 -0800
Subject: [M3devel] performance problems in CM3 Text operations on
	AMD64_LINUX
In-Reply-To: <COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
References: <20140104185511.9000A1A2080@async.async.caltech.edu>,
	<52CF0E43.2030701@lcwb.coop>
	<COL130-W5205CC733E268A6D3C441E6B30@phx.gbl>
Message-ID: <20140110152711.036F21A208F@async.async.caltech.edu>


>Mika=2C how did you find this?Control-c every so often and look at callstac=
>k?Or a lot of stepping?

"Poor man's profiling" -- since profiling is usually broken in Modula-3.  It worked back with SRC M3 but that was a while ago :-)

1. run program in gdb
2. close eyes
3. count to three
4. hit ctrl-C
5. open eyes

(repeat a few times)

"close eyes" is important!


From dragisha at m3w.org  Sun Jan 12 09:49:23 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 09:49:23 +0100
Subject: [M3devel] Conversion to Git
Message-ID: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>

Hi,

I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.

For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.

alexb
darko
dbenavid
dragisha
eiserlohpp
hosking
hudson
jkrell
khaeusler
kschleiser
m3
mand
micha
mika
mm
neels
pmckinna
rcoleburn
rforb
rillig
rodney
sk
stsp
thielema
uamoore
wagner
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/330f89f9/attachment-0002.sig>

From mika at async.caltech.edu  Sun Jan 12 18:53:17 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:53:17 -0800
Subject: [M3devel] CM3 crashing
Message-ID: <20140112175317.9F0031A2080@async.async.caltech.edu>

Hi m3devel,

Anyone have an idea what's going on here?

I have program that compiles with PM3 (FreeBSD4):

...
new source -> compiling ../src/Main.m3
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

new opaque info -> recompiling HParseStd.m3
...

With CM3 at head (AMD64_LINUX (userthreads)), the following happens:

--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure


***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
***

Abort

(gdb) run -x
Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
--- building in AMD64_LINUX ---

echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
new source -> compiling Main.m3
"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory
"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure

Program received signal SIGSEGV, Segmentation fault.
0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
36            u.get_info (RInfo);  
(gdb) where
#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading variable>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variable>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error reading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:1289
#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading variable>) at ../src/misc/CG.m3:2547
#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable>) at ../src/values/Procedure.m3:358
#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) at ../src/values/Value.m3:61
#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variable>) at ../src/exprs/ProcExpr.m3:96
#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:145
#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading variable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error reading variable>) at ../src/values/Formal.m3:550
#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading variable>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:459
#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:131
#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:431
#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at ../src/types/UserProc.m3:91
#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at ../src/exprs/CallExpr.m3:314
#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../src/exprs/Expr.m3:138
#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading variable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker.m3:540
#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading variable>) at ../src/stmts/ReturnStmt.m3:55
#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:345
#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable>) at ../src/stmts/IfStmt.m3:105
#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variable>) at ../src/stmts/WithStmt.m3:150
#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error reading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error reading variable>, M3_AcxOUs_case_label=<error reading variable>, 
    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:339
#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading variable>) at ../src/stmts/TypeCaseStmt.m3:284
#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading variable>) at ../src/stmts/BlockStmt.m3:103
#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>) at ../src/stmts/Stmt.m3:115
#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading variable>) at ../src/values/Procedure.m3:577
#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading variable>) at ../src/values/Procedure.m3:541
#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading variable>) at ../src/misc/ProcBody.m3:170
#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error reading variable>) at ../src/misc/ProcBody.m3:69
#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:905
#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable>) at ../src/values/Module.m3:837
#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading variable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error reading variable>) at ../src/misc/M3Front.m3:66
#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variable>) at ../src/Builder.m3:1705
#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading variable>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error reading variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m3:1025
#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_AicXUJ_shared=<error reading variable>, 
    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading variable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variable>) at ../src/QMachine.m3:1773
#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_Bd56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_file=<error reading variable>) at ../src/M3Build.m3:679
#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>)
    at ../src/QMachine.m3:546
#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading variable>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../src/Main.m3:231
#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe390) at _m3main.c:22
(gdb) 

Looks like something very basic...





From mika at async.caltech.edu  Sun Jan 12 18:58:30 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 09:58:30 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175317.9F0031A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
Message-ID: <20140112175830.C51881A2080@async.async.caltech.edu>

Sigh...

    188 PROCEDURE PutErr (u: U;  a, b, c: TEXT := NIL) =
    189   BEGIN
    190     u.note_error ("********* M3CG_Check ERROR *********** " & a & b & c);
    191     u.child.comment ("********* M3CG_Check ERROR *********** ", a, b, c);
    192     INC (u.n_errors);
    193   END PutErr;

Ah.... ok....

"../src/Main.m3", line 129: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 131: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"../src/Main.m3", line 133: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL**
"

The code is:

              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
            |
              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
            |
              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
 
I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 

mika writes:
>Hi m3devel,
>
>Anyone have an idea what's going on here?
>
>I have program that compiles with PM3 (FreeBSD4):
>
>...
>new source -> compiling ../src/Main.m3
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>new opaque info -> recompiling HParseStd.m3
>...
>
>With CM3 at head (AMD64_LINUX (userthreads)), the following happens:
>
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>
>***
>*** runtime error:
>***    Attempt to reference an illegal memory location.
>***    file "../src/thread/POSIX/ThreadPosix.m3", line 1073
>***
>
>Abort
>
>(gdb) run -x
>Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x
>[Thread debugging using libthread_db enabled]
>Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>--- building in AMD64_LINUX ---
>
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl
>echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl
>new source -> compiling Main.m3
>"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca
>tor.OutOfMemory
>"../src/Main.m3", line 15: warning: exception is never raised: OSError.E
>"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted
>"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure
>
>Program received signal SIGSEGV, Segmentation fault.
>0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable>, M
>3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>36            u.get_info (RInfo);  
>(gdb) where
>#0  0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=<error reading variable
>>, M3_Bd56fi_u=<error reading variable>) at ../src/text/TextCat.m3:36
>#1  0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=<error reading varia
>ble>, M3_Bd56fi_a=<error reading variable>, M3_Bd56fi_b=<error reading variabl
>e>, M3_Bd56fi_c=<error reading variable>) at ../src/M3CG_Check.m3:190
>#2  0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=<error reading
> variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3:513
>#3  0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=<error re
>ading variable>, M3_BgUkgw_p=<error reading variable>) at ../src/M3CG_Check.m3
>:1289
>#4  0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=<error reading varia
>ble>) at ../src/misc/CG.m3:2547
>#5  0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=<error reading variable
>>) at ../src/values/Procedure.m3:358
>#6  0x000000000054f81f in Value__Load (M3_EjfEr4_t=<error reading variable>) a
>t ../src/values/Value.m3:61
>#7  0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=<error reading variab
>le>) at ../src/exprs/ProcExpr.m3:96
>#8  0x0000000000596873 in Expr__Compile (M3_ES44mX_t=<error reading variable>)
> at ../src/exprs/Expr.m3:145
>#9  0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>, M3_ES44mX_proc=<error readi
>ng variable>) at ../src/values/Formal.m3:550
>#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=<error reading varia
>ble>, M3_EjfEr4_formal=<error reading variable>, M3_ES44mX_actual=<error readi
>ng variable>) at ../src/values/Formal.m3:459
>#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:131
>#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#13 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=<error reading var
>iable>, M3_ES44mX_actual=<error reading variable>) at ../src/values/Formal.m3:
>431
>#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=<error reading variable>) at 
>../src/types/UserProc.m3:91
>#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=<error reading variable>) at .
>./src/exprs/CallExpr.m3:314
>#17 0x0000000000596825 in Expr (M3_ES44mX_t=<error reading variable>) at ../sr
>c/exprs/Expr.m3:138
>#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=<error reading va
>riable>, M3_AicXUJ_fromFinally=<error reading variable>) at ../src/misc/Marker
>.m3:540
>#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=<error reading vari
>able>) at ../src/stmts/ReturnStmt.m3:55
>#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:345
>#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=<error reading variable
>>) at ../src/stmts/IfStmt.m3:105
>#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=<error reading variab
>le>) at ../src/stmts/WithStmt.m3:150
>#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=<error re
>ading variable>, M3_CUZbO1_c=<error reading variable>, M3_CFkyBH_ref=<error re
>ading variable>, M3_AcxOUs_case_label=<error reading variable>, 
>    M3_AcxOUs_exit_label=<error reading variable>, M3_EIM114_done=<error readi
>ng variable>) at ../src/stmts/TypeCaseStmt.m3:339
>#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=<error reading va
>riable>) at ../src/stmts/TypeCaseStmt.m3:284
>#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=<error reading varia
>ble>) at ../src/stmts/BlockStmt.m3:103
>#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=<error reading variable>)
> at ../src/stmts/Stmt.m3:115
>#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=<error reading varia
>ble>) at ../src/values/Procedure.m3:577
>#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=<error reading vari
>able>) at ../src/values/Procedure.m3:541
>#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=<error reading varia
>ble>) at ../src/misc/ProcBody.m3:170
>#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=<error readin
>g variable>) at ../src/misc/ProcBody.m3:69
>#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=<error reading va
>riable>) at ../src/values/Module.m3:905
>#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=<error reading variable
>>) at ../src/values/Module.m3:837
>#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218
>#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=<error reading var
>iable>, M3_CT7GjM_env=<error reading variable>, M3_BySCJy_options=<error readi
>ng variable>) at ../src/misc/M3Front.m3:66
>#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=<error reading variable>
>, M3_AXLf8B_u=<error reading variable>, M3_Bd56fi_object=<error reading variab
>le>) at ../src/Builder.m3:1705
>#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1359
>#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=<error reading varia
>ble>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1247
>#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=<error reading vari
>able>, M3_AXLf8B_u=<error reading variable>) at ../src/Builder.m3:1077
>#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=<error readi
>ng variable>, M3_Cw4bpV_schedule=<error reading variable>) at ../src/Builder.m
>3:1025
>#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=<error reading
> variable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<erro
>r reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
>    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:336
>#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading var
>iable>, M3_EEuw3X_units=<error reading variable>, M3_C1FTrk_sys_libs=<error re
>ading variable>, M3_AicXUJ_shared=<error reading variable>, 
>    M3_An02H2_m=<error reading variable>) at ../src/Builder.m3:28
>#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading va
>riable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
>#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=<error reading varia
>ble>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
>#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#64 0x00000000004d9152 in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>, M3_AicXUJ_from_code=<error reading variab
>le>) at ../src/QMachine.m3:1773
>#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=<error reading variable>, M3_B
>d56fi_path=<error reading variable>) at ../src/QMachine.m3:1744
>#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=<error reading variable>, M3_Bd
>56fi_file=<error reading variable>) at ../src/M3Build.m3:679
>#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=<error reading va
>riable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1054
>#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=<error reading variabl
>e>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading
> variable>, M3_AicXUJ_outer=<error reading variable>)
>    at ../src/QMachine.m3:546
>#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=<error reading variable>
>) at ../src/QMachine.m3:422
>#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=<error reading varia
>ble>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
>#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=<error reading variable>, M3
>_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
>#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, 
>M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
>#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117
>#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at
> ../src/Main.m3:231
>#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading va
>riable>) at ../src/runtime/common/RTLinker.m3:408
>#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading varia
>ble>) at ../src/runtime/common/RTLinker.m3:115
>#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variab
>le>) at ../src/runtime/common/RTLinker.m3:124
>#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe3
>90) at _m3main.c:22
>(gdb) 
>
>Looks like something very basic...
>
>


From hosking at cs.purdue.edu  Sun Jan 12 21:30:34 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sun, 12 Jan 2014 15:30:34 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140112175830.C51881A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
Message-ID: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>

Are you saying passing these used to work in PM3?
Sounds like a front-end bug.  I?m curious what changed to break it.

On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

> 
> The code is:
> 
>              HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab)
>            |
>              HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab)
>            |
>              HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab)
> 
> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... 



Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484







From dragisha at m3w.org  Sun Jan 12 22:45:32 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 12 Jan 2014 22:45:32 +0100
Subject: [M3devel] Conversion to Git
In-Reply-To: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
References: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org>
Message-ID: <E4D3225F-F1C5-4660-9EFB-6EA49218733F@m3w.org>

Thanks to everybody who send me infomation, esp. to Peter Eiserloh who gave me information + idea how to fill rest. Missing are emails from some Elego people and I expect I will have that soon.

I will keep you all posted on conversion status. Current version is on Github (dragisha/cm3), and I am checking how it matches with local CVS checkouts.

On 12 Jan 2014, at 09:49, Dragi?a Duri? <dragisha at m3w.org> wrote:

> Hi,
> 
> I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion.
> 
> For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data.
> 
> alexb
> darko
> dbenavid
> dragisha
> eiserlohpp
> hosking
> hudson
> jkrell
> khaeusler
> kschleiser
> m3
> mand
> micha
> mika
> mm
> neels
> pmckinna
> rcoleburn
> rforb
> rillig
> rodney
> sk
> stsp
> thielema
> uamoore
> wagner
> --
> Dragi?a Duri?
> dragisha at m3w.org
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140112/d532a608/attachment-0002.sig>

From mika at async.caltech.edu  Mon Jan 13 01:25:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 12 Jan 2014 16:25:39 -0800
Subject: [M3devel] CM3 crashing
In-Reply-To: <AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
Message-ID: <20140113002539.EC4EE1A2080@async.async.caltech.edu>

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>


From hosking at cs.purdue.edu  Mon Jan 13 16:46:36 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 10:46:36 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <09749A02-B9B9-4C74-8CB4-73379E91385B@cs.purdue.edu>


On Jan 12, 2014, at 7:25 PM, mika at async.caltech.edu wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

Interesting.  Needs exploration.  This should be filed as a bug.  The workaround is to wrap the builtin calls (Word.Xor, etc.) in a local procedure so that you have an addressable procedure.

> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?

Yes, they are inlined, but there *is* an implementation in Word.m3 (generated from GenWord.mg).

> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?

It must be the threads implementation, since the same collector is used with both pthreads and user threads, and I believe that user threads don?t suffer from this problem.  My intuition is that it may be a problem with getting stack roots on particular platforms, probably because the stack bounds are not being computed correctly for a given architecture, or perhaps because reference alignment is not being observed correctly.  For example, is it possible on x86_64 to have references stored in the stack aligned on 32-bit instead of 64-bit boundaries?

I notice that RTThread.i3 says the pointer alignment is BYTESIZE(ADDRESS) on all platforms.  Once upon a time the alignment was platform-dependent, so I wonder if we lost something somewhere with all the backend and porting work.  It may be that the backend needs to be instructed to align all references appropriately in the stacks.

One way to test all this might be to see if the thread stress tester causes things to break on older 32-bit platforms, where I expect all pointers will be aligned properly.

Also, I?ve not inspected the code in ThreadPThreadC.c recently (it?s changed since I last looked), but I wonder if the stack bounds are being computed correctly so that the collector can find all the stack roots.

I?d really like to get to the bottom of this problem.

> Tony Hosking writes:
>> Are you saying passing these used to work in PM3?
>> Sounds like a front-end bug.  I=92m curious what changed to break it.
>> 
>> On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
>> 
>>> =20
>>> The code is:
>>> =20
>>>             HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Xor), ab)
>>>           |
>>>             HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.Or), ab)
>>>           |
>>>             HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>> Word.And), ab)
>>> =20
>>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>> 
>> 
>> 
>> Antony Hosking | Associate Professor | Computer Science | Purdue =
>> University
>> 305 N. University Street | West Lafayette | IN 47907 | USA
>> Mobile +1 765 427 5484
>> 
>> 
>> 
>> 



From hosking at cs.purdue.edu  Mon Jan 13 17:03:38 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 11:03:38 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <EA2F5D60-CBC8-47A1-BC34-B3981DAFE4BD@cs.purdue.edu>

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:

> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 
> 
> Regards Peter
> 
> 
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/202c6d98/attachment-0002.html>

From rcolebur at SCIRES.COM  Mon Jan 13 20:16:41 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:16:41 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy

Randy C. Coleburn, CISSP-ISSEP, GCED
Corporate Information Security Officer (CISO)
Scientific Research Corporation
2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339
voice: (770) 989-9464, email: RColeburn at SciRes.com<mailto:RColeburn at SciRes.com>, fax: (770) 989-9497

[logo_SRC]
"Technology Driven-Customer Focused"

Quality Policy:  "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements."

CONFIDENTIALITY NOTICE:  This email constitutes an electronic communication within the meaning of the Electronic Communications Privacy Act, 18 U.S.C. 2510, and its disclosure is strictly limited to the named recipient(s) intended by the sender of this message.  This email, and any attachments, may contain confidential and/or proprietary information of Scientific Research Corporation.  If you are not a named recipient, any copying, using, disclosing or distributing to others the information in this email and attachments is STRICTLY PROHIBITED.  If you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts or hard copies of the email and attachments.

EXPORT COMPLIANCE NOTICE:  This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR).  Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer.  In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization.  By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:


Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 11339 bytes
Desc: image002.jpg
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/49ca426d/attachment-0002.jpg>

From rcolebur at SCIRES.COM  Mon Jan 13 20:55:44 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 19:55:44 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/3dbd395c/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Jan 13 21:02:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 13 Jan 2014 15:02:57 -0500
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL>
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <227E8C61-B070-49FF-8AD5-E23420E9C1B0@cs.purdue.edu>

Is this from the head of source?
Also, please run with @M3paranoidgc flag.


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 
>  
> C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386
>  
> C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2013-12-17 17:28:52
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> Here is the output:
>  
> C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> This last assert repeats indefinitely until you press CTRL-C.
>  
> --Randy
>  
>  
> From: Coleburn, Randy 
> Sent: Monday, January 13, 2014 2:17 PM
> To: Tony Hosking; Peter McKinna
> Cc: m3devel
> Subject: Re: [M3devel] CM3 crashing
>  
> Tony et al:
>  
> The thread test program fails for me on 64-bit Windows 7.
>  
> I?ve listed output from a couple of runs below:
>  
> C:\cm3\Sandbox>cm3 --version
> Critical Mass Modula-3 version d5.9.0
>   last updated: 2010-07-21
>   compiled: 2014-01-07 06:55:14
>   configuration: C:\cm3\bin\cm3.cfg
>   host: NT386
>   target: NT386
>  
> FIRST RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\runtime\common\RTCollector.m3", line 418
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
> ***
>  
>                ?this last assertion repeats until you press CTRL-C to abort the program?
>  
> SECOND RUN:
>  
> C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
> Writing file...done
> Creating read threads...done
> Creating fork threads...done
> Creating alloc threads...done
> Creating lock threads...done
> running...printing oldest/median age/newest
> ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
> ....
>  
> ***
> *** runtime error:
> ***    Attempt to reference an illegal memory location.
> ***    pc = 0x776f22d2
> ***
>  
> Stack trace:
>    FP         PC      Procedure
> ---------  ---------  -------------------------------
> 0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
> 0xa6f9d0  0x776f22d2  <???>
> 0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
> 0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
> 0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
> 0xa6fb90  0x76db336a  <???>
> 0xa6fbd0  0x7770bf32  <???>
> .........  .........  ... more frames ...
>  
> Maybe this info will be useful in tracking down the problem.
>  
> --Randy
>  
>  
> From: Tony Hosking [mailto:hosking at cs.purdue.edu] 
> Sent: Monday, January 13, 2014 11:04 AM
> To: Peter McKinna
> Cc: m3devel
> Subject: EXT:Re: [M3devel] CM3 crashing
>  
> Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
> In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.
>  
> As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?
>  
> If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.
>  
> On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:
>  
> 
> Hey,
> 
>   I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 
> My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
> It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
> 
> Regards Peter
>  
>  
> 
> On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:
> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).
> 
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
> 
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
> 
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/c56a7e47/attachment-0002.html>

From rcolebur at SCIRES.COM  Mon Jan 13 21:13:55 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 20:13:55 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:


I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/6198676d/attachment-0002.html>

From rcolebur at SCIRES.COM  Mon Jan 13 22:02:52 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Mon, 13 Jan 2014 21:02:52 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>

Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484<tel:%2B1%20765%20427%205484>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/eea82060/attachment-0002.html>

From jay.krell at cornell.edu  Tue Jan 14 00:22:12 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 13 Jan 2014 23:22:12 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>,
	<0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W762DFE2FF1E39C9147F90EE6BC0@phx.gbl>

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: 
 
  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html   
 
 
 And disappointed. Cooperative suspend will fix this someday. 
 
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: hosking at cs.purdue.edu
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.



 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the
 finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:



 





Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much
 always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok
 to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the
 alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.



It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 





 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/45717d65/attachment-0002.html>

From rcolebur at SCIRES.COM  Tue Jan 14 01:11:07 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Tue, 14 Jan 2014 00:11:07 +0000
Subject: [M3devel] CM3 crashing
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?

Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Monday, January 13, 2014 6:22 PM
To: Coleburn, Randy; Tony
Cc: m3devel
Subject: EXT:RE: [M3devel] CM3 crashing

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html


 And disappointed. Cooperative suspend will fix this someday.


 - Jay

________________________________
From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
To: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Mon, 13 Jan 2014 21:02:52 +0000
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
Subject: Re: [M3devel] CM3 crashing
Tony:

Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option.

These may give some more clues.

I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious.

By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.

Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I'll commit a fix for that momentarily.

But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing.

RUN #1:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...

RUN #2:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Coleburn, Randy
Sent: Monday, January 13, 2014 3:14 PM
To: Tony Hosking
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony:

Yes, it is from the HEAD as of 17 December 2013.

Using the @M3paranoidgc flag,

On 32-bit Windows XP I get:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

On 64-bit Windows 7 I get:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.

***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***

***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.

--Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 3:03 PM
To: Coleburn, Randy
Cc: Peter McKinna; m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Is this from the head of source?
Also, please run with @M3paranoidgc flag.

On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

I've also tested on 32-bit Windows XP and the thread test program crashes there also.

C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386

C:\cm3\Sandbox\cm3\m3-libs>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2013-12-17 17:28:52
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

Here is the output:

C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

This last assert repeats indefinitely until you press CTRL-C.

--Randy


From: Coleburn, Randy
Sent: Monday, January 13, 2014 2:17 PM
To: Tony Hosking; Peter McKinna
Cc: m3devel
Subject: Re: [M3devel] CM3 crashing

Tony et al:

The thread test program fails for me on 64-bit Windows 7.

I've listed output from a couple of runs below:

C:\cm3\Sandbox>cm3 --version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-07 06:55:14
  configuration: C:\cm3\bin\cm3.cfg
  host: NT386
  target: NT386

FIRST RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    Attempt to reference an illegal memory location.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 418
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***

               ...this last assertion repeats until you press CTRL-C to abort the program...

SECOND RUN:

C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
....

***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x776f22d2
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0xa6f9d0  0x776f22d2  <???>
0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3
0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3
0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0xa6fb90  0x76db336a  <???>
0xa6fbd0  0x7770bf32  <???>
.........  .........  ... more frames ...

Maybe this info will be useful in tracking down the problem.

--Randy


From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Monday, January 13, 2014 11:04 AM
To: Peter McKinna
Cc: m3devel
Subject: EXT:Re: [M3devel] CM3 crashing

Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).
In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem.

As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?

If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed.

On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com<mailto:peter.mckinna at gmail.com>> wrote:

Hey,
  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.
Regards Peter


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu<mailto:mika at async.caltech.edu>> wrote:
Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).

I figured there aren't actually implementations of Word.Xor, Word.Or,
and Word.And as procedures but they get inlined somehow?

BTW, any idea about what's wrong with pthreads?  Do you think the issue
is with the garbage collector or with the threads, off the top of your
head?

Tony Hosking writes:
>Are you saying passing these used to work in PM3?
>Sounds like a front-end bug.  I=92m curious what changed to break it.
>
>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu<mailto:mika at async.caltech.edu> wrote:
>
>>=20
>> The code is:
>>=20
>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Xor), ab)
>>            |
>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.Or), ab)
>>            |
>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
>Word.And), ab)
>>=20
>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
>
>
>
>Antony Hosking | Associate Professor | Computer Science | Purdue =
>University
>305 N. University Street | West Lafayette | IN 47907 | USA
>Mobile +1 765 427 5484
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/7dd888c2/attachment-0002.html>

From jay.krell at cornell.edu  Tue Jan 14 01:21:49 2014
From: jay.krell at cornell.edu (Jay K)
Date: Tue, 14 Jan 2014 00:21:49 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W14788FF4A8D081652DA79DE6BF0@phx.gbl>

I don't know. We can try writing a stress test for it. It looks like one is available.
Agreed -- work out the bugs in the native Windows and Posix systems.
 
 - Jay
 
From: rcolebur at SCIRES.COM
To: jay.krell at cornell.edu; hosking at cs.purdue.edu
Date: Tue, 14 Jan 2014 00:11:07 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing









Jay:
 
The article you reference is dated 13 Nov 2010.
Do you know whether Microsoft has done anything about this problem over the past 3 years?
 
Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext.
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Monday, January 13, 2014 6:22 PM

To: Coleburn, Randy; Tony

Cc: m3devel

Subject: EXT:RE: [M3devel] CM3 crashing


 

 I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here:


 

  http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html  


 

 

 And disappointed. Cooperative suspend will fix this someday. 

 

 

 - Jay

 






From: 
rcolebur at SCIRES.COM

To: hosking at cs.purdue.edu

Date: Mon, 13 Jan 2014 21:02:52 +0000

CC: m3devel at elegosoft.com

Subject: Re: [M3devel] CM3 crashing

Tony:
 
Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option.
 
These may give some more clues.
 
I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks
very suspicious.
 
By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0.
 
Oops, now I see the problem:  there is a bug in the PutInt procedure.  The test (IF c > 10) should read instead (IF c >= 10).  I?ll
 commit a fix for that momentarily.
 
But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing.
 
RUN #1:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3
***
 
Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0x7cfd60   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3
0x7cfda8   0xc286e0  Init + 0x79 in ..\src\rw\FileRd.m3
0x7cfdd4   0xc2865d  Open + 0x4d in ..\src\rw\FileRd.m3
0x7cff0c   0xc21350  RApply + 0x160 in ..\src\Main.m3
0x7cff48   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3
0x7cff54  0x76db336a  <???>
0x7cff94  0x7770bf32  <???>
.........  .........  ... more frames ...
 
RUN #2:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose
Writing file...done
Creating read threads...
   read=0
   read=1
   read=2
done
Creating fork threads...
   fork=9
   fork=0
   fork=11
done
Creating alloc threads...
   alloc=15
   alloc=16
   alloc=17
done
Creating lock threads...
   lock=21
   lock=22
   lock=23
done
running...printing oldest/median age/newest
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 534 loops.
   read Thread 1 completed 606 loops.
   read Thread 2 completed 398 loops.
   fork Thread 9 completed 8 loops.
   fork Thread 10 completed 8 loops.
   fork Thread 11 completed 8 loops.
   alloc Thread 15 completed 18296 loops.
   alloc Thread 16 completed 44871 loops.
   alloc Thread 17 completed 79125 loops.
   lock Thread 21 completed 3845080 loops.
   lock Thread 22 completed 2537613 loops.
   lock Thread 23 completed 4506151 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 790 loops.
   read Thread 1 completed 786 loops.
   read Thread 2 completed 675 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 22091 loops.
   alloc Thread 16 completed 47532 loops.
   alloc Thread 17 completed 68901 loops.
  lock Thread 21 completed 4560731 loops.
   lock Thread 22 completed 3440795 loops.
   lock Thread 23 completed 6433538 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 725 loops.
   read Thread 1 completed 705 loops.
   read Thread 2 completed 617 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18560 loops.
   alloc Thread 16 completed 44487 loops.
   alloc Thread 17 completed 87219 loops.
   lock Thread 21 completed 3769840 loops.
   lock Thread 22 completed 3037581 loops.
   lock Thread 23 completed 6097922 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 825 loops.
   read Thread 1 completed 802 loops.
   read Thread 2 completed 669 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 16877 loops.
   alloc Thread 16 completed 53277 loops.
   alloc Thread 17 completed 77986 loops.
   lock Thread 21 completed 4087218 loops.
   lock Thread 22 completed 3116357 loops.
   lock Thread 23 completed 6466214 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 684 loops.
   read Thread 1 completed 686 loops.
   read Thread 2 completed 648 loops.
   fork Thread 9 completed 10 loops.
   fork Thread 10 completed 9 loops.
   fork Thread 11 completed 10 loops.
   alloc Thread 15 completed 19717 loops.
   alloc Thread 16 completed 47797 loops.
   alloc Thread 17 completed 78110 loops.
   lock Thread 21 completed 4580435 loops.
   lock Thread 22 completed 2974247 loops.
   lock Thread 23 completed 5843850 loops.
..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)
   read Thread 0 completed 748 loops.
   read Thread 1 completed 755 loops.
   read Thread 2 completed 639 loops.
   fork Thread 9 completed 9 loops.
   fork Thread 10 completed 10 loops.
   fork Thread 11 completed 9 loops.
   alloc Thread 15 completed 18037 loops.
   alloc Thread 16 completed 46293 loops.
   alloc Thread 17 completed 82560 loops.
   lock Thread 21 completed 4210008 loops.
   lock Thread 22 completed 2708008 loops.
   lock Thread 23 completed 6139723 loops.
..
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Coleburn, Randy


Sent: Monday, January 13, 2014 3:14 PM

To: Tony Hosking

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing


 
Tony:
 
Yes, it is from the HEAD as of 17 December 2013.
 
Using the
@M3paranoidgc flag,
 
On
32-bit Windows XP I get:
 
C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    Attempt to reference an illegal memory location.
***    pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
On
64-bit Windows 7 I get:
 
C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.
 
***
*** runtime error:
***    An array subscript was out of range.
***    file "..\src\rw\FileRd.m3", line 83
***
 
***
*** runtime error:
***    An enumeration or subrange value was out of range.
***    file "..\src\runtime\common\RTType.m3", line 71
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
 
***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823
***
This last assert repeats indefinitely until you press CTRL-C.
 
--Randy
 


From: Tony Hosking [mailto:hosking at cs.purdue.edu]


Sent: Monday, January 13, 2014 3:03 PM

To: Coleburn, Randy

Cc: Peter McKinna; m3devel

Subject: EXT:Re: [M3devel] CM3 crashing


 
Is this from the head of source?

Also, please run with @M3paranoidgc flag.


 


On Jan 13, 2014, at 2:55 PM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

 


I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. 


 


C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386


 


C:\cm3\Sandbox\cm3\m3-libs>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2013-12-17 17:28:52


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


Here is the output:


 


C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


This last assert repeats indefinitely until you press CTRL-C.


 


--Randy


 


 




From: Coleburn,
 Randy 

Sent: Monday, January 13, 2014 2:17 PM

To: Tony Hosking; Peter McKinna

Cc: m3devel

Subject: Re: [M3devel] CM3 crashing




 


Tony et al:


 


The thread test program fails for me on 64-bit Windows 7.


 


I?ve listed output from a couple of runs below:


 


C:\cm3\Sandbox>cm3 --version


Critical Mass Modula-3 version d5.9.0


  last updated: 2010-07-21


  compiled: 2014-01-07 06:55:14


  configuration: C:\cm3\bin\cm3.cfg


  host: NT386


  target: NT386


 


FIRST RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\runtime\common\RTCollector.m3", line 418


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


***


*** runtime error:


***    <*ASSERT*> failed.


***    file "..\src\thread\WIN32\ThreadWin32.m3", line 823


***


 


               ?this last assertion repeats until you press CTRL-C to abort the program?


 


SECOND RUN:


 


C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help


Writing file...done


Creating read threads...done


Creating fork threads...done


Creating alloc threads...done


Creating lock threads...done


running...printing oldest/median age/newest


..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0)


....


 


***


*** runtime error:


***    Attempt to reference an illegal memory location.


***    pc = 0x776f22d2


***


 


Stack trace:


   FP         PC      Procedure


---------  ---------  -------------------------------


0xa6f9a8   0xc6afeb  SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3


0xa6f9d0  0x776f22d2  <???>


0xa6f9e8   0xc4a39b  LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fa10   0xc299f8  GetChar + 0x28 in ..\src\rw\Rd.m3


0xa6fb48   0xc213a8  RApply + 0x1b8 in ..\src\Main.m3


0xa6fb84   0xc4b71b  ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3


0xa6fb90  0x76db336a  <???>


0xa6fbd0  0x7770bf32  <???>


.........  .........  ... more frames ...


 


Maybe this info will be useful in tracking down the problem.


 


--Randy


 


 




From: Tony
 Hosking [mailto:hosking at cs.purdue.edu] 

Sent: Monday, January 13, 2014 11:04 AM

To: Peter McKinna

Cc: m3devel

Subject: EXT:Re: [M3devel] CM3 crashing




 



Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?).




In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive)
 to see if we can spot a problem.




 




As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC.  Combined with my assumption that user
 threads work fine, it would seem to point the finger at stack scanning.  Can someone confirm?




 




If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without
 keeping the collector informed.




 





On Jan 12, 2014, at 10:54 PM, Peter McKinna <peter.mckinna at gmail.com> wrote:


 




Hey,


  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it
 with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with
 code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the
 read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only
 way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to
 process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.


It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs.


Regards Peter



 




 


On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:


Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?





Tony Hosking writes:

>Are you saying passing these used to work in PM3?



>Sounds like a front-end bug.  I=92m curious what changed to break it.



>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>



>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =



>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>







 




 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140114/2693506b/attachment-0002.html>

From peter.mckinna at gmail.com  Mon Jan 13 04:54:08 2014
From: peter.mckinna at gmail.com (Peter McKinna)
Date: Mon, 13 Jan 2014 14:54:08 +1100
Subject: [M3devel] CM3 crashing
In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>
	<20140112175830.C51881A2080@async.async.caltech.edu>
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>
Message-ID: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>

Hey,

  I was trying to get a handle on that problem last year. The threadtest
program is really a stress tester of the collector/allocator with pthreads.
If you run it with just the tests read and alloc you pretty much always get
a crash. If you run them with paranoidgc it will crash in the heap checker.
Tony thought it was a clear cut  problem of the roots of some ref not being
found on a thread stack. I mucked around with code to get some output and
the stacks looked ok to me but I could be wrong. All stacks are checked
whilst the threads are blocked in a signal handler, the design of which
looks fine as far as I can tell. This test is characterised by some slow
threads (the read threads) and a bunch of fast threads (the alloc threads).
Even if you modify the test to have only one read thread the problem
occurs. I have had misgivings about mixing signals and threads having been
bitten many years ago, but really this is the only way the collector can
get its raw refs to check.
My guess is its a subtle timing or lock problem maybe a lurking bug in the
collector itself. One thing I noticed is that in
ThreadPThread__ProcessStopped  the second call to   p(context, ((char
*)context) + sizeof(ucontext_t)); according to the comment is to process
the registers. But the registers should already be on the stack and anyway
this call is a partial duplicate of the previous one.
It would be good to raise the priority on this problem. Trust in the
collector has always been at the heart of m3 programs.

Regards Peter



On Mon, Jan 13, 2014 at 11:25 AM, <mika at async.caltech.edu> wrote:

> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a
> reason to switch to CM3).
>
> I figured there aren't actually implementations of Word.Xor, Word.Or,
> and Word.And as procedures but they get inlined somehow?
>
> BTW, any idea about what's wrong with pthreads?  Do you think the issue
> is with the garbage collector or with the threads, off the top of your
> head?
>
> Tony Hosking writes:
> >Are you saying passing these used to work in PM3?
> >Sounds like a front-end bug.  I=92m curious what changed to break it.
> >
> >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:
> >
> >>=20
> >> The code is:
> >>=20
> >>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Xor), ab)
> >>            |
> >>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.Or), ab)
> >>            |
> >>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =
> >Word.And), ab)
> >>=20
> >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20
> >
> >
> >
> >Antony Hosking | Associate Professor | Computer Science | Purdue =
> >University
> >305 N. University Street | West Lafayette | IN 47907 | USA
> >Mobile +1 765 427 5484
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140113/9bfaddc0/attachment-0002.html>

From jay.krell at cornell.edu  Wed Jan 15 06:01:16 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 05:01:16 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>,
	<20140112175830.C51881A2080@async.async.caltech.edu>,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>
Message-ID: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>

Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/7fcd3a4f/attachment-0002.html>

From jay.krell at cornell.edu  Wed Jan 15 07:48:30 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 15 Jan 2014 06:48:30 +0000
Subject: [M3devel] CM3 crashing
In-Reply-To: <COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
References: <20140112175317.9F0031A2080@async.async.caltech.edu>, ,
	<20140112175830.C51881A2080@async.async.caltech.edu>, ,
	<AB0685D5-31AC-423E-BB8F-30AED3B781CC@cs.purdue.edu>, ,
	<20140113002539.EC4EE1A2080@async.async.caltech.edu>, ,
	<CAEy6uxofhqi1Bwh=8a4kWwo17mhchEVODZpvqVw2hp=d=wSd3g@mail.gmail.com>,
	<COL130-W3108C4B07141BE282D247EE6BE0@phx.gbl>
Message-ID: <COL130-W18EA0744A6E2E144FEB51AE6BE0@phx.gbl>

ok, I augmented the test a bit and commited it to CVS and I can see it fail now.
Does the test make sense to people?
 
Thanks,
 - Jay

 
From: jay.krell at cornell.edu
To: peter.mckinna at gmail.com; mika at async.caltech.edu
Date: Wed, 15 Jan 2014 05:01:16 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing




Here is my failed attempt to demonstrate the wow64  bug.
Make sense?
Repros for anyone? I only tried Windows 8 so far.
 
 
#undef NDEBUG
#include <assert.h>
#include <stddef.h>
#include <windows.h>
void* _AddressOfReturnAddress(void);
volatile size_t expected;
#define PAGE 4096
__declspec(noinline) void X(void)
{
  expected = (size_t)_AddressOfReturnAddress();
  Sleep(1);
  expected = 0;
}
__declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); }
__declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); }
__declspec(noinline) unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
  {
    F1();
    F2();
  }
  return 0;
}
int __cdecl wmain()
{
  unsigned __int64 count = 0;
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  CONTEXT context;
  ZeroMemory(&context, sizeof(context));
  context.ContextFlags = CONTEXT_CONTROL;
  //while (count++ < 1000000)
  while (1)
  {
    SuspendThread(thread);
    GetThreadContext(thread, &context);
    assert(expected == 0 || (context.Esp && context.Esp < expected));
    ResumeThread(thread);
  }
}

 - Jay


 
Date: Mon, 13 Jan 2014 14:54:08 +1100
From: peter.mckinna at gmail.com
To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] CM3 crashing

Hey,

  I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut  problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. 

My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped  the second call to   p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one.

It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. 

Regards Peter




On Mon, Jan 13, 2014 at 11:25 AM,  <mika at async.caltech.edu> wrote:

Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3).



I figured there aren't actually implementations of Word.Xor, Word.Or,

and Word.And as procedures but they get inlined somehow?



BTW, any idea about what's wrong with pthreads?  Do you think the issue

is with the garbage collector or with the threads, off the top of your

head?



Tony Hosking writes:

>Are you saying passing these used to work in PM3?

>Sounds like a front-end bug.  I=92m curious what changed to break it.

>

>On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote:

>

>>=20

>> The code is:

>>=20

>>              HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Xor), ab)

>>            |

>>              HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.Or), ab)

>>            |

>>              HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, =

>Word.And), ab)

>>=20

>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20

>

>

>

>Antony Hosking | Associate Professor | Computer Science | Purdue =

>University

>305 N. University Street | West Lafayette | IN 47907 | USA

>Mobile +1 765 427 5484

>

>

>

>


 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140115/cb7c9be3/attachment-0002.html>

From jay.krell at cornell.edu  Sat Jan 18 07:22:53 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 18 Jan 2014 06:22:53 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <COL130-W549E4FE51DBF493AF61101E6BB0@phx.gbl>

 This program also doesn't behave as expected, native, nothing to do with wow64. 
 Anyone else please confirm: 
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it  
   3) their not being met -- stuff gets printed 
 This is in the CVS repository, scratch/wow64stack/sync2.cpp 
 I am following up further. 
 Maybe we should get cooperative suspend really going? 
Thank you.
 - Jay 
 
#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value; 
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/26ed6280/attachment-0002.html>

From rcolebur at SCIRES.COM  Sat Jan 18 08:11:45 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Sat, 18 Jan 2014 07:11:45 +0000
Subject: [M3devel] Win32 SuspendThread?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

I'm very concerned about the threading not working properly on both 32-bit and 64-bit Windows.

The Thread Test program crashes for me on both platforms.

I haven't tried your new test program yet.

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Saturday, January 18, 2014 1:23 AM
To: m3devel
Subject: EXT:[M3devel] Win32 SuspendThread?

 This program also doesn't behave as expected, native, nothing to do with wow64.
 Anyone else please confirm:
   1) my expectations -- it should never print anything
   2) their importance  -- garbage collector depends on it
   3) their not being met -- stuff gets printed
 This is in the CVS repository, scratch/wow64stack/sync2.cpp
 I am following up further.
 Maybe we should get cooperative suspend really going?
Thank you.
 - Jay

#include <stdio.h>
#include <windows.h>
volatile long value;
unsigned long __stdcall Thread(PVOID parameter)
{
  while (1)
   InterlockedIncrement(&value);
  return 0;
}
int __cdecl main()
{
  HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
  UINT i = 0;
  while (1)
  {
    i += 1;
    if (SuspendThread(thread) == (DWORD)-1)
    {
      printf("suspend failed %X\n", GetLastError());
      Sleep(1);
      continue;
    }
    volatile long a = value;
    volatile long b = value;
    if (a != b)
    {
     printf("%d %d %d %d\n", i, a, b, b - a);
    }
    ResumeThread(thread);
  }
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/ee809938/attachment-0002.html>

From dragisha at m3w.org  Sat Jan 18 09:24:11 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 18 Jan 2014 09:24:11 +0100
Subject: [M3devel] Cooperative suspend?
Message-ID: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>

Jay,

What exactly do you mean? Isn?t it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?

dd
--
Dragi?a Duri?
dragisha at m3w.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/08eadd0f/attachment-0002.sig>

From mika at async.caltech.edu  Sun Jan 19 04:40:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sat, 18 Jan 2014 19:40:06 -0800
Subject: [M3devel] how is this possible?
Message-ID: <20140119034006.34F511A208F@async.async.caltech.edu>


Any hypothesis is welcome...

(311)truffles:~/t/btc/hparse/src>../AMD64_LINUX/hparse and.scm 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../AMD64_LINUX/BDDSchemeStubs.m3", line 619
***

Abort
(312)truffles:~/t/btc/hparse/src>

    613 PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } =
    614   BEGIN
    615     IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END;
    616     IF NOT ISTYPE(x,BDD.T) THEN
    617       RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
    618     END;
    619     RETURN x
    620   END ToModula_BDD_T;


The object is of type BDD.Root (I checked the TYPECODE), which is
declared thus:


TYPE
  Root = T BRANDED Brand & " Root" OBJECT
    mu : MUTEX; (* as yet unused *)
    id : CARDINAL;
    tab : BDDTripleHash.T;
    cache : ARRAY Op OF BDDTripleHash.T;
  END;

SchemeObject.T is the same as REFANY

I'm not even sure where to start debugging this.  The code "should work"...

Indeed:

> (rttype-typecode a)
144
> (RTBrand.GetByTC 144)
"BDD 0.1 Root"
> (RTBrand.GetByTC 143)
"BDD 0.1"
> (RTType.Supertype 143)
4

(4 is REFANY)

> (BDD.And a a)
EXCEPTION! RuntimeError! An explicit or implicit NARROW() operation failed.
> a-and-a
<Modula-3 object : BDDImpl.Root, BRANDED BDD 0.1 Root>
> (rttype-typecode a-and-a)
144


     Mika




From jay.krell at cornell.edu  Sun Jan 19 02:02:56 2014
From: jay.krell at cornell.edu (Jay)
Date: Sat, 18 Jan 2014 17:02:56 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>

For native I'm less worried now. For wow64 I'm still worried & hope to follow up further.

 - Jay

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:

> Jay:
>  
> I?m very concerned about the threading not working properly on both 32-bit and 64-bit Windows.
>  
> The Thread Test program crashes for me on both platforms.
>  
> I haven?t tried your new test program yet.
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Saturday, January 18, 2014 1:23 AM
> To: m3devel
> Subject: EXT:[M3devel] Win32 SuspendThread?
>  
>  This program also doesn't behave as expected, native, nothing to do with wow64. 
>  Anyone else please confirm: 
>    1) my expectations -- it should never print anything
>    2) their importance  -- garbage collector depends on it  
>    3) their not being met -- stuff gets printed 
>  This is in the CVS repository, scratch/wow64stack/sync2.cpp 
>  I am following up further. 
>  Maybe we should get cooperative suspend really going? 
> Thank you.
>  - Jay 
>  
> #include <stdio.h>
> #include <windows.h>
> volatile long value;
> unsigned long __stdcall Thread(PVOID parameter)
> {
>   while (1)
>    InterlockedIncrement(&value);
>   return 0;
> }
> int __cdecl main()
> {
>   HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0);
>   UINT i = 0;
>   while (1)
>   {
>     i += 1;
>     if (SuspendThread(thread) == (DWORD)-1)
>     {
>       printf("suspend failed %X\n", GetLastError());
>       Sleep(1);
>       continue;
>     }
>     volatile long a = value; 
>     volatile long b = value;
>     if (a != b)
>     {
>      printf("%d %d %d %d\n", i, a, b, b - a);
>     }
>     ResumeThread(thread);
>   }
> }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140118/21485aaa/attachment-0002.html>

From mika at async.caltech.edu  Sun Jan 19 10:41:39 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 01:41:39 -0800
Subject: [M3devel] something wrong with ISTYPE ...
Message-ID: <20140119094139.3917E1A208F@async.async.caltech.edu>


Hi m3devel,

There is definitely something wrong with ISTYPE.  The problem I reported earlier goes away if I change the code like so:

(416)truffles:~/t/btc/hparse/AMD64_LINUX>diff -c BDDSchemeStubs.m3.old BDDSchemeStubs.m3
*** BDDSchemeStubs.m3.old       2014-01-19 01:13:33.000000000 -0800
--- BDDSchemeStubs.m3   2014-01-19 01:36:52.000000000 -0800
***************
*** 613,622 ****
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     IF NOT ISTYPE(x,BDD.T) THEN
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END;
!     RETURN x
    END ToModula_BDD_T;
  
  
--- 613,624 ----
  PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 
    BEGIN
      IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 
!     TYPECASE x OF
!       BDD.T(xx) => RETURN xx
!     ELSE
        RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x))
!     END
! 
    END ToModula_BDD_T;

This shouldn't be possible... it's the same object as before, with the
same return type.  I don't get an exception (I shouldn't either)---the
return just behaves normally and the program works.  Why doesn't it work
with ISTYPE but only with TYPECASE?

BTW, this is an old bug.  Same behavior with ancient PM3.  Where to look
in the code?

    Mika


From adacore at marino.st  Sun Jan 19 11:03:27 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 11:03:27 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Message-ID: <52DBA2EF.4020101@marino.st>

Hi all,

I've been trying to port cm3 to DragonFly x86-64.
I'm able to build cm3cg on DragonFly, and use it to build libraries
(e.g. libm3core) that were targeted for AMD64_FREEBSD.

However, when I set the HOST to AMD64_FREEBSD and TARGET to
AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
target machine: TARGET = AMD64_DRAGONFLY" (context below)

=========================================================
> /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> ROOT        = /work/lang/modula3/work/modula3-5.8.6
> CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> M3GDB       = yes
> M3OSTYPE    = POSIX
> TARGET      = AMD64_DRAGONFLY
> GCC_BACKEND = yes
> INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> GREP        = egrep
> GMAKE       = gmake
> TMPDIR      = /var/tmp
> EXE         = 
> SL          = /
> TAR         = tar
> CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> CM3VERSION  = 5.8.6
> CM3VERSIONNUM = 050806
> CM3LASTCHANGED = 2010-04-11
> FIND = /usr/bin/find
> EGREP = egrep
> === package m3-libs/m3core ===
>  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> --- building in AMD64_DRAGONFLY ---
> 
> ignoring ../src/m3overrides
> 
> 
> Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> 
>  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> *** [assembler-code] Error code 1

=========================================================

I thought I added all the target definitions but apparently I've missed
something.  I also realize the current repo looks a lot different, but I
wanted to use the last official release.

I've attached a concatenated file of patches showing what I've changed.
 Could somebody suggest what data is missing that causes the error above?

Regards,
John
-------------- next part --------------
--- m3-libs/libm3/src/os/POSIX/m3makefile.orig	2009-07-24 10:02:40.000000000 +0000
+++ m3-libs/libm3/src/os/POSIX/m3makefile
@@ -84,6 +84,7 @@ local readonly DefaultOSName = {
    "MIPS64_OPENBSD" : "OpenBSD",
    "SPARC64_SOLARIS" : "Solaris",
    "I386_OPENBSD" : "OpenBSD",
+   "AMD64_DRAGONFLY" : "DragonFly",
    "AMD64_FREEBSD" : "FreeBSD",
    "PA32_HPUX" : "HPUX",
    "PA64_HPUX" : "HPUX",
@@ -92,6 +93,7 @@ local readonly DefaultOSName = {
    "AIX" : "AIX",
    "CYGWIN" : "Cygwin",
    "DARWIN" : "Darwin",
+   "DRAGONFLY" : "DragonFly",
    "FREEBSD" : "FreeBSD",
    "HPUX" : "HPUX",
    "INTERIX" : "Interix",
@@ -156,6 +158,7 @@ local readonly DefaultArch = {
    "SPARC64_SOLARIS" : "sparc64",
    "I386_OPENBSD" : "i686",
    "AMD64_FREEBSD" : "AMD64",
+   "AMD64_DRAGONFLY" : "AMD64",
    "PA32_HPUX" : "hppa",
    "PA64_HPUX" : "hppa64",
 
--- m3-libs/m3core/src/Csupport/m3makefile.orig	2009-07-24 10:51:01.000000000 +0000
+++ m3-libs/m3core/src/Csupport/m3makefile
@@ -11,6 +11,7 @@ readonly _CSupportPieces = {
 % "ALPHA_OSF"  : [ "little-endian" ],
 % "AIX386"     : [ "little-endian" ],
   "AMD64_DARWIN": [ "little-endian" ],
+  "AMD64_DRAGONFLY": [ "little-endian" ],
   "AMD64_FREEBSD": [ "little-endian" ],
   "AMD64_OPENBSD": [ "little-endian" ],
   "AMD64_NETBSD": [ "little-endian" ],
--- m3-libs/m3core/src/float/m3makefile.orig	2009-07-24 10:51:24.000000000 +0000
+++ m3-libs/m3core/src/float/m3makefile
@@ -13,6 +13,7 @@ _FloatPieces = {
 % "ALPHA_OSF"  : _float_le,
 % "AIX386"     : _float_le,
   "AMD64_DARWIN": _float_le,
+  "AMD64_DRAGONFLY": _float_le,
   "AMD64_FREEBSD": _float_le,
   "AMD64_OPENBSD": _float_le,
   "AMD64_NETBSD": _float_le,
--- m3-libs/m3core/src/m3core.h.orig	2010-03-28 10:07:45.000000000 +0000
+++ m3-libs/m3core/src/m3core.h
@@ -207,7 +207,8 @@ typedef INTEGER m3_uid_t;
 #define PTHREAD_TO_M3(x)   ((m3_pthread_t)(size_t)(x))
 #define PTHREAD_FROM_M3(x) ((pthread_t)(size_t)(x))
 
-#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__)
+#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__OpenBSD__) || defined(__DragonFly__)
 #define HAS_STAT_FLAGS
 #endif
 
--- m3-libs/m3core/src/platforms.quake.orig	2010-03-28 10:09:07.000000000 +0000
+++ m3-libs/m3core/src/platforms.quake
@@ -1,5 +1,6 @@
 _TARGET_OS = {
   "AMD64_DARWIN"  : "DARWIN",
+  "AMD64_DRAGONFLY" : "DRAGONFLY",
   "AMD64_FREEBSD" : "FREEBSD",
   "AMD64_LINUX"   : "LINUX",
   "AMD64_NETBSD"  : "NETBSD",
@@ -32,6 +33,7 @@ _TARGET_ENDIAN = {
 % This is not computed based on architecture
 % because some architectures vary, e.g. PPC_NT is little endian, PPC_DARWIN is big.
   "AMD64_DARWIN"    : "LITTLE",
+  "AMD64_DRAGONFLY" : "LITTLE",
   "AMD64_FREEBSD"   : "LITTLE",
   "AMD64_LINUX"     : "LITTLE",
   "AMD64_NETBSD"    : "LITTLE",
--- m3-libs/m3core/src/runtime/POSIX/RTSignalC.c.orig	2010-04-16 12:20:56.000000000 +0000
+++ m3-libs/m3core/src/runtime/POSIX/RTSignalC.c
@@ -167,7 +167,7 @@ static size_t GetPC(void* xcontext)
 #elif defined(__NetBSD__)
     _UC_MACHINE_PC(context)
 
-#elif defined(__FreeBSD__)
+#elif defined(__FreeBSD__) || defined(__DragonFly__)
 #if defined(__amd64)
     context->uc_mcontext.mc_rip
 #elif defined(__i386)
--- m3-libs/m3core/src/runtime/common/Compiler.tmpl.orig	2009-07-13 09:09:35.000000000 +0000
+++ m3-libs/m3core/src/runtime/common/Compiler.tmpl
@@ -21,7 +21,7 @@ readonly Compiler_i3 = [
   "               SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD,",
   "               SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD,",
   "               PA32_HPUX, PA64_HPUX, ARM_DARWIN, I386_INTERIX,",
-  "               AMD64_NETBSD, AMD64_OPENBSD };",
+  "               AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY };",
   "",
   "CONST",
   "  ThisOS       = OS." & OS_TYPE & ";",
--- m3-libs/m3core/src/thread.quake.orig	2009-12-20 11:01:02.000000000 +0000
+++ m3-libs/m3core/src/thread.quake
@@ -12,6 +12,7 @@ readonly NO_USER_THREADS =
     "AMD64_FREEBSD":1,
     "AMD64_OPENBSD":1,
     "AMD64_NETBSD":1,
+    "AMD64_DRAGONFLY":1,
     "AMD64_LINUX":1,
     "ARM_LINUX":1,
     "MIPS32_IRIX":1,
--- m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c.orig	2009-12-21 05:35:34.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c
@@ -2,7 +2,7 @@
 /* All rights reserved.                                            */
 /* See the file COPYRIGHT-PURDUE for a full description.           */
 
-#ifndef __FreeBSD__
+#if !defined(__FreeBSD__) && !defined(__DragonFly__)
 
 /* avoid empty file */
 
--- m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c.orig	2009-12-12 14:48:10.000000000 +0000
+++ m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c
@@ -23,7 +23,7 @@
 #endif
 #endif
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 #define M3_DIRECT_SUSPEND
 #endif
 
@@ -177,7 +177,7 @@ void ThreadPThread__sem_post(void)      
 void ThreadPThread__sem_getvalue(void)  { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 void ThreadPThread__sigsuspend(void)    { M3_DIRECT_SUSPEND_ASSERT_FALSE }
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
 
 int
 ThreadPThread__SuspendThread (m3_pthread_t mt)
--- m3-libs/m3core/src/time/POSIX/m3makefile.orig	2009-07-24 07:26:15.000000000 +0000
+++ m3-libs/m3core/src/time/POSIX/m3makefile
@@ -11,6 +11,7 @@ readonly _DateImpls = {
 % "ALPHA_OSF"  : "DateBsd",
 % "AIX386"     : "DateBsd",
   "AMD64_DARWIN": "DateBsd",
+  "AMD64_DRAGONFLY": "DateBsd",
   "AMD64_FREEBSD": "DateBsd",
   "AMD64_OPENBSD": "DateBsd",
   "AMD64_NETBSD": "DateBsd",
@@ -63,6 +64,7 @@ readonly _DateImpls = {
 % "AIX"     : "DateBsd",
   "CYGWIN"  : "DatePosix",
   "DARWIN"  : "DateBsd",
+  "DRAGONFLY" : "DataBsd",
   "FREEBSD" : "DateBsd",
   "HPUX"    : "DatePosix",
   "INTERIX" : "DatePosix",
--- m3-libs/m3core/src/unix/Common/UtimeC.c.orig	2009-12-11 11:18:34.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/UtimeC.c
@@ -56,7 +56,8 @@ struct timeval
 #endif
 }
 
-#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__APPLE__)
+#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) \
+ || defined(__APPLE__) || defined(__DragonFly__)
 #define M3BSD
 #endif
 
--- m3-libs/m3core/src/unix/Common/Uucontext.c.orig	2009-06-29 19:20:40.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/Uucontext.c
@@ -29,6 +29,7 @@ see http://www.opengroup.org/onlinepubs/
     || defined(__linux) \
     || defined(__NetBSD__) \
     || defined(__hpux) \
+    || defined(__DragonFly__) \
     || defined(__FreeBSD__) \
 
 int Uucontext__getcontext(ucontext_t* context)
--- m3-libs/m3core/src/unix/Common/m3makefile.orig	2009-07-19 00:28:17.000000000 +0000
+++ m3-libs/m3core/src/unix/Common/m3makefile
@@ -81,7 +81,7 @@ if equal (TARGET, "MIPS64_OPENBSD") or e
         or equal (TARGET, "PA32_HPUX") or equal (TARGET, "PA64_HPUX")
         or equal (TARGET, "NT386") or equal (TARGET, "AMD64_FREEBSD")
         or equal (TARGET, "AMD64_OPENBSD") or equal (TARGET, "AMD64_NETBSD")
-        or equal (TARGET, "I386_INTERIX")
+        or equal (TARGET, "I386_INTERIX") or equal (TARGET, "AMD64_DRAGONFLY")
 
     Interface("Uucontext")
 
--- m3-libs/m3core/src/unix/m3makefile.orig	2009-07-28 18:49:13.000000000 +0000
+++ m3-libs/m3core/src/unix/m3makefile
@@ -6,6 +6,7 @@ readonly _UnixPieces = {
 % "AIX386"          : [ "aix-ps2-1-2",                             "uin-common" ],
 % "ALPHA_OSF"       : [ "osf-1.generic", "osf-1.ALPHA_OSF",        "uin-common" ],
   "AMD64_DARWIN"    : [ "darwin-common", "darwin-amd64",           "uin-len" ],
+  "AMD64_DRAGONFLY" : [ "freebsd-common",                          "uin-len" ],
   "AMD64_FREEBSD"   : [ "freebsd-common",                          "uin-len" ],
   "AMD64_LINUX"     : [ "linux-common",                            "uin-common" ],
   "AMD64_NETBSD"    : [ "netbsd-common",                           "uin-len" ],
--- m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY.orig	2014-01-16 23:14:23.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY
@@ -0,0 +1,9 @@
+readonly TARGET = "AMD64_DRAGONFLY"
+readonly GNU_PLATFORM = "amd64-dragonfly" % "cpu-os" string for GNU
+
+m3back_flags = "-gstabs+ -m64 -fPIC -funwind-tables"
+SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC"
+readonly SYSTEM_ASM = "as -64"
+
+include("AMD64.common")
+include("DragonFly.common")
--- m3-sys/cminstall/src/config-no-install/DragonFly.common.orig	2014-01-16 23:18:36.000000000 +0000
+++ m3-sys/cminstall/src/config-no-install/DragonFly.common
@@ -0,0 +1,24 @@
+readonly TARGET_OS = "DRAGONFLY"
+
+GNU_MAKE = "gmake"
+
+include("Unix.common")
+
+SYSTEM_LIBS{"ODBC"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lodbc" ]
+SYSTEM_LIBS{"POSTGRES95"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lpq" ]
+SYSTEM_LIBS{"X11"} = ["-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ]
+
+DRAGONFLY_CC_APPEND = " -z origin"
+                    & " -z now"
+                    & " -Bsymbolic"
+
+DRAGONFLY_LD_APPEND = DRAGONFLY_CC_APPEND
+                    & " -Wl,--fatal-warnings"
+                    & " -Wl,--warn-common"
+                    & " -Wl,-rpath,/usr/local/lib"
+                    & " -Wl,-rpath,\\$ORIGIN/../lib "
+
+SYSTEM_LD = SYSTEM_CC & DRAGONFLY_LD_APPEND
+SYSTEM_CC = SYSTEM_CC & DRAGONFLY_CC_APPEND
+
+include("gnuld.common")
--- m3-sys/m3cc/src/boot.py.orig	2008-05-19 13:09:03.000000000 +0000
+++ m3-sys/m3cc/src/boot.py
@@ -76,7 +76,7 @@ if SearchPath("gcc"):
     " + env)
 
   Run(gmake + "all-gmp all-mpfr all-libcpp all-libdecnumber \
-    all-build-libiberty all-libiberty configure-gcc")
+    all-build-libiberty all-libiberty configure-libcpp configure-gcc")
 
   os.chdir("gcc")
 
--- m3-sys/m3cc/src/m3makefile.orig	2010-05-14 07:00:52.000000000 +0000
+++ m3-sys/m3cc/src/m3makefile
@@ -577,7 +577,7 @@ end
 if GCC42
     m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
 else
-    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log])
+    m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber configure-libcpp | tee -a " & Log])
 end
 
 %
--- m3-sys/m3cc/src/platforms.quake.orig	2009-07-29 17:11:08.000000000 +0000
+++ m3-sys/m3cc/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "i686-apple-darwin8",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd7",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3gdb/src/platforms.quake.orig	2010-04-21 17:06:38.000000000 +0000
+++ m3-sys/m3gdb/src/platforms.quake
@@ -13,6 +13,7 @@ readonly Platform_info = {
   "ALPHA_OSF" : "alpha-dec-osf1",
   "ALPHA_VMS" : "alpha-vms",
   "AMD64_DARWIN" : "amd64-apple-darwin8.7.1",
+  "AMD64_DRAGONFLY" : "amd64-dragonfly",
   "AMD64_FREEBSD" : "amd64-freebsd",
   "AMD64_LINUX" : "amd64-pc-linux-gnu",
   "AMD64_NETBSD" : "amd64-netbsd",
--- m3-sys/m3middle/src/Target.i3.orig	2010-02-01 14:05:35.000000000 +0000
+++ m3-sys/m3middle/src/Target.i3
@@ -33,7 +33,7 @@ TYPE
     AMD64_DARWIN, AMD64_LINUX, SPARC32_LINUX, SPARC64_LINUX,
     SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD, SPARC64_SOLARIS,
     I386_OPENBSD, AMD64_FREEBSD, PA32_HPUX, PA64_HPUX, ARM_DARWIN,
-    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, Undefined
+    I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY, Undefined
   };
 
 CONST
@@ -89,7 +89,8 @@ CONST
     (* 48 *) "ARM_DARWIN",
     (* 49 *) "I386_INTERIX",
     (* 50 *) "AMD64_NETBSD",
-    (* 51 *) "AMD64_OPENBSD"
+    (* 51 *) "AMD64_OPENBSD",
+    (* 52 *) "AMD64_DRAGONFLY"
   };
 
 CONST
--- m3-sys/m3middle/src/Target.m3.orig	2010-02-01 15:06:05.000000000 +0000
+++ m3-sys/m3middle/src/Target.m3
@@ -307,6 +307,7 @@ PROCEDURE Init (system: TEXT; in_OS_name
 
     |  Systems.AMD64_NETBSD,
        Systems.AMD64_OPENBSD,
+       Systems.AMD64_DRAGONFLY,
        Systems.AMD64_FREEBSD =>
                  Jumpbuf_size              := 16_60 * Char.size;
 
--- m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c.orig	2009-04-12 10:31:23.000000000 +0000
+++ m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c
@@ -1,11 +1,11 @@
-#ifdef __FreeBSD__
+#if defined __FreeBSD__) || defined(__DragonFly__)
 #include <sys/types.h>
 #include <libutil.h>
 #endif
 
 void m3_setproctitle(const char *title)
 {
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) || defined(__DragonFly__)
     setproctitle("%s", title);
 #endif
 }
--- scripts/boot-cm3-makefile-lib.tmpl.orig	2005-04-07 22:35:07.000000000 +0000
+++ scripts/boot-cm3-makefile-lib.tmpl
@@ -20,9 +20,10 @@ FILES_C  := $(wildcard *.c)
 FILES_S  := $(wildcard *.s)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C) $(FILES_S))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 
--- scripts/boot-cm3-makefile-prog.tmpl.orig	2003-07-10 22:16:05.000000000 +0000
+++ scripts/boot-cm3-makefile-prog.tmpl
@@ -19,9 +19,10 @@ FILES_MS := $(wildcard *.ms)
 FILES_C  := $(wildcard *.c)
 BASES_IC := $(basename $(FILES_IC))
 BASES_MC := $(basename $(FILES_MC))
+BASES_MS := $(basename $(FILES_MS))
 BASES_C  := $(basename $(FILES_C))
 OBJ_IO   := $(addsuffix .io, $(BASES_IC))
-OBJ_MO   := $(addsuffix .mo, $(BASES_MC))
+OBJ_MO   := $(addsuffix .mo, $(BASES_MS))
 OBJ_O    := $(addsuffix .o, $(BASES_C))
 OBJECTS  := $(OBJ_IO) $(OBJ_MO) $(OBJ_O)
 LIBFILES := $(shell for f in $(LIBS); do if [ -f "$${f}" ] ; then  echo $${f}; fi; done)
--- scripts/pack-crossbuild.sh.orig	2010-02-25 14:02:18.000000000 +0000
+++ scripts/pack-crossbuild.sh
@@ -23,7 +23,7 @@ if [ -z "$1" ] ; then
 fi
 CROSS_TARGET=$1
 BUILDARGS=''
-M3CONFIG=${ROOT}/m3-sys/cm3/src/config/${CROSS_TARGET}
+M3CONFIG=${ROOT}/m3-sys/cminstall/src/config-no-install/${CROSS_TARGET}
 export M3CONFIG
 . "$ROOT/scripts/pkginfo.sh"
 . "$ROOT/scripts/pkgcmds.sh"
@@ -38,25 +38,7 @@ P="${P} m3back"
 P="${P} m3front"
 P="${P} m3quake"
 P="${P} cm3"
-P="${P} m3scanner"
-P="${P} m3tools"
-P="${P} m3cgcat"
-P="${P} m3cggen"
-[ "${M3GDB}" = yes ] && P="${P} m3gdb"
 P="${P} m3bundle"
-P="${P} mklib"
-P="${P} fix_nl"
-P="${P} libdump"
-P="${P} bitvector"
-P="${P} digraph"
-P="${P} parseparams"
-P="${P} realgeometry"
-P="${P} set"
-P="${P} slisp"
-P="${P} sortedtableextras"
-P="${P} table-list"
-P="${P} tempfiles"
-[ "${HAVE_TCL}" = "yes" ] && P="${P} tcl"
 
 USAGE="
   `basename $0` cross_target

From jay.krell at cornell.edu  Sun Jan 19 12:35:54 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:35:54 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBA2EF.4020101@marino.st>
References: <52DBA2EF.4020101@marino.st>
Message-ID: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>

C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/013940cc/attachment-0002.html>

From jay.krell at cornell.edu  Sun Jan 19 12:40:17 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 11:40:17 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>

Your change looks about right.
 
Did you first "upgrade" and existing install, with these changes, and then cross?
That is what you must do.
 
 - Jay
 
From: jay.krell at cornell.edu
To: adacore at marino.st; m3devel at elegosoft.com
Subject: RE: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
Date: Sun, 19 Jan 2014 11:35:54 +0000




C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized target machin
e: TARGET = ", s.target);
 
 
 Please don't bother with the last release. 
 Please use the current CVS version. 
 And please use the C backend.
 Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?
 
 
 Please use this as a guide for files to consider changing:
 C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
m3-comm\udp\test\src\m3makefile
m3-libs\libm3\src\os\POSIX\m3makefile
m3-libs\m3core\src\float\m3makefile
m3-libs\m3core\src\thread\m3makefile
m3-libs\m3core\src\thread\PTHREAD\m3makefile
m3-libs\m3core\src\unix\m3makefile
m3-sys\m3cc\src\m3makefile
m3-sys\m3gdb\src\m3makefile
m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
m3-tools\cvsup\suplib\src\m3makefile
m3-tools\cvsup\suptcp\src\POSIX\m3makefile
 
C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
  m3-libs\libm3\src\uid\POSIX\getMID.c  
  m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c 
  m3-libs\m3core\src\Csupport\Common\s_lroundl.c 
 m3-libs\m3core\src\float\C99\FloatModeC.c 
   m3-libs\m3core\src\runtime\common\RTProcessC.c  
  m3-libs\m3core\src\runtime\POSIX\RTSignalC.c   
  m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c 
  m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c 
  m3-libs\m3core\src\unix\Common\Uin.c
  m3-libs\patternmatching\src\libglob\fnmatch.c 

 
esp. m3core and libm3.
 
 
 If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you.
 But I realize there is much value in me NOT doing it. 
 I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. 
 
 
 - Jay

 
> Date: Sun, 19 Jan 2014 11:03:27 +0100
> From: adacore at marino.st
> To: m3devel at elegosoft.com
> Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> Hi all,
> 
> I've been trying to port cm3 to DragonFly x86-64.
> I'm able to build cm3cg on DragonFly, and use it to build libraries
> (e.g. libm3core) that were targeted for AMD64_FREEBSD.
> 
> However, when I set the HOST to AMD64_FREEBSD and TARGET to
> AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized
> target machine: TARGET = AMD64_DRAGONFLY" (context below)
> 
> =========================================================
> > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core
> > ROOT        = /work/lang/modula3/work/modula3-5.8.6
> > CM3_ROOT    = /work/lang/modula3/work/modula3-5.8.6
> > M3GDB       = yes
> > M3OSTYPE    = POSIX
> > TARGET      = AMD64_DRAGONFLY
> > GCC_BACKEND = yes
> > INSTALLROOT = /work/lang/modula3/work/stage/usr/local
> > PKGSDB      = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS
> > GREP        = egrep
> > GMAKE       = gmake
> > TMPDIR      = /var/tmp
> > EXE         = 
> > SL          = /
> > TAR         = tar
> > CM3ROOT     = /work/lang/modula3/work/modula3-5.8.6
> > CM3VERSION  = 5.8.6
> > CM3VERSIONNUM = 050806
> > CM3LASTCHANGED = 2010-04-11
> > FIND = /usr/bin/find
> > EGREP = egrep
> > === package m3-libs/m3core ===
> >  +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  +++
> > --- building in AMD64_DRAGONFLY ---
> > 
> > ignoring ../src/m3overrides
> > 
> > 
> > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY
> > 
> >  *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS  failed ***
> > *** [assembler-code] Error code 1
> 
> =========================================================
> 
> I thought I added all the target definitions but apparently I've missed
> something.  I also realize the current repo looks a lot different, but I
> wanted to use the last official release.
> 
> I've attached a concatenated file of patches showing what I've changed.
>  Could somebody suggest what data is missing that causes the error above?
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/1e7c8efe/attachment-0002.html>

From adacore at marino.st  Sun Jan 19 12:53:53 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 12:53:53 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
Message-ID: <52DBBCD1.7020904@marino.st>

On 1/19/2014 12:35, Jay K wrote:
> C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3
> m3-sys\cm3\src\Builder.m3:      Msg.FatalError (NIL, "unrecognized
> target machin
> e: TARGET = ", s.target);
>  
>  
>  Please don't bother with the last release.
>  Please use the current CVS version.

But the last release is what is in FreeBSD ports, the basis of both
FreeBSD and DragonFly.  And ports doesn't build from repos normally,
although there is a facility to pull directly from github, but I don't
like that method myself.

If there was an official downloadable snapshot or release candidate, I
could use that.  Since 2010 was the last release, is another one coming
soon?


>  And please use the C backend.
>  Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3?

Yes, that was one of the many patches in the attachment I send in the
previous mail.


>  Please use this as a guide for files to consider changing:
>  C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile
> m3-comm\udp\test\src\m3makefile
> m3-libs\libm3\src\os\POSIX\m3makefile
> m3-libs\m3core\src\float\m3makefile
> m3-libs\m3core\src\thread\m3makefile
> m3-libs\m3core\src\thread\PTHREAD\m3makefile
> m3-libs\m3core\src\unix\m3makefile
> m3-sys\m3cc\src\m3makefile
> m3-sys\m3gdb\src\m3makefile
> m3-tools\cvsup\suplib\src\FreeBSD\m3makefile
> m3-tools\cvsup\suplib\src\m3makefile
> m3-tools\cvsup\suptcp\src\POSIX\m3makefile

The only ones I haven't patched already are
m3-libs\m3core\src\thread\PTHREAD\m3makefile and the cvsup ones.  Those
wouldn't cause the error.  I also assume this is the top of the repo.


> C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c
>   m3-libs\libm3\src\uid\POSIX\getMID.c 
>   m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c
>   m3-libs\m3core\src\Csupport\Common\s_lroundl.c
>  m3-libs\m3core\src\float\C99\FloatModeC.c 
>    m3-libs\m3core\src\runtime\common\RTProcessC.c 
>   m3-libs\m3core\src\runtime\POSIX\RTSignalC.c  
>   m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c
>   m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c
>   m3-libs\m3core\src\unix\Common\Uin.c
>   m3-libs\patternmatching\src\libglob\fnmatch.c
> 
>  
> esp. m3core and libm3.

I haven't seen freebsd in these, so must be a difference between release
and repo.



>  If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it
> for you.
>  But I realize there is much value in me NOT doing it.
>  I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok.

Because AMD64_DRAGONFLY is too long?
I'd prefer to keep it "AMD64_DRAGONFLY" because it matches the other
targets and because of all the work I've already done.


I'm really, really close to getting cm3 bootstrapped on DragonFly with
5.8.6.  I believe if this "M3_BOOTSTRAP = true" version would compile, I
could build cm3 and libraries on DragonFly with the cm3cg that's already
built on it.  Once I have a fully native compiler, then I can update to
the latest repo, especially if some kind of snapshot is released.

Thanks,
John


From jay.krell at cornell.edu  Sun Jan 19 13:00:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:00:01 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
Message-ID: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>

Windows uses SuspendThread.
I'll double check.


Cooperative suspend would NOT involve sending a "signal" to threads, in the Posix sense.
Nor would it use SuspendThread.


The model for cooperative suspend has been described by Tony and goes something like this:

There is a list of threads maintained by the m3 runtime.
  Not by enumerating OS threads.
The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.


When a thread wants to collect garbage, it either walks the thread list and sets all the booleans/enums to "suspend requested", or, more likely, it sets one global boolean.

Now, there are three likely methods to set the global boolean.
 1 - it is a global writable variable 
 2 - it is a global writable function pointer 
 3 - it is a page with protection 


Code frequently polls this global boolean, in one of the following fashions
 1 - read and test the value of the variable 
 2 - call the function pointer 
 3 - read the page (just a byte or a word) ignoring the contents 
 4 - call a function to do any of the above, esp #1 


#3 is probably fastest but my least favorite. It has lingering portability problems and is slowest when it comes time to actually suspend -- catch the exception and for purposes of debugging, you really don't want a design that raises exceptions like that.


When does the polling occur?
The polling is generated by the compiler.
It is done at function entry and/or exit and at branches or at least backward branches or at least branches that are continuing loops or such or at the starts of loops or somesuch.


I believe Java uses cooperative suspend and I think it uses the page protection mechanism.
It isn't portable e.g. to systems without an MMU.
Though of course you could abstract it out.


I'm glossing over details, but you know all the code we have for signals and semaphores? It goes away.
Our portability then would drive one more significant notch toward that of "hello world".

In my mine, we have about two pieces to do to gain "hello world" level of portability
  - cooperative suspend, not easy  
  - jmpbuf size removal, should be easy  

and some small matters
  - endianness, particularly in the libary 
  - word size 



 - Jay


From: dragisha at m3w.org
Date: Sat, 18 Jan 2014 09:24:11 +0100
To: m3devel at elegosoft.com
Subject: [M3devel] Cooperative suspend?

Jay,
What exactly do you mean? Isn't it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend?
dd

--Dragi?a Duri?dragisha at m3w.org



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/88389edf/attachment-0002.html>

From adacore at marino.st  Sun Jan 19 13:09:37 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:09:37 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>
Message-ID: <52DBC081.1070401@marino.st>

On 1/19/2014 12:40, Jay K wrote:
> Your change looks about right.
>  
> Did you first "upgrade" and existing install, with these changes, and
> then cross?
> That is what you must do.


It's probably easier that I show you.
I've attached "Makefile.local" (I added the .txt extension just now to
help out some mail clients) which is my recipe.

So I run three targets in a freshly patched workarea:
1) make cross-compiler
2) make assembler-code (called from cross-archive)
3) make cross-archive

In the cross-compiler target, the FreeBSD bootstrap compiler builds
m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.

The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
Everything is cleaned, then the cross compiler builds everything again
(including libm3core and libm3) along with m3objfile, sysutils, and
patternmatching.

Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
cm3cg is cm3cg-AMD64_DRAGONFLY.

I would not be surprised is there is a logic flaw or two in "make
cross-compiler".  The "how to port CM3" tutorial was slightly out of
date and some of the instructions were a bit ambiguous so this was a
best guess.

Regards,
John
-------------- next part --------------
# This makefile fragment is automatically included by bsd.port.mk
# It is separate because it contains maintenance targets not used in
# normal package building, but the recipes are too value to lose.  So
# we keep them with the port, but out of the way of regular users.
#
# $FreeBSD$


##
#
# The "create-bootstrap" target will assemble a new bootstrap based on the
# contents of the stage directory, which means it has to be run after the
# "stage" target.
#
##

NEWBOOTDIR=	${WRKDIR}/new-bootstrap/bootstrap
NEWBOOTNAME=	m3-bootstrap.${MARCH}.${OPSYS:U}.${OSREL:S/.//}.tar.bz2
BSCONTENTS=	bin/cm3 bin/cm3cg bin/m3bundle bin/mklib etc/modula3 \
		lib/libm3core.* lib/libm3.* lib/libsysutils.* \
		lib/libpatternmatching.* pkg/m3core pkg/libm3 pkg/sysutils \
		pkg/patternmatching pkg/m3middle pkg/m3objfile pkg/m3linker \
		pkg/m3back pkg/m3front pkg/m3quake pkg/cm3 pkg/mklib

create-bootstrap:
	@${RM} -rf ${NEWBOOTDIR}
	@${MKDIR} ${NEWBOOTDIR}/bin ${NEWBOOTDIR}/lib \
	          ${NEWBOOTDIR}/pkg ${NEWBOOTDIR}/etc
.for X in ${BSCONTENTS}
	@${CP} -a ${STAGEDIR}${PREFIX}/${X} ${NEWBOOTDIR}/${X:H}/
.endfor
	${ECHO} "INSTALL_ROOT = path() & \"/..\"" \
		>  ${NEWBOOTDIR}/bin/cm3.cfg
	${ECHO} "include(path() & \"/../etc/modula3/${M3TARGET}\")" \
		>> ${NEWBOOTDIR}/bin/cm3.cfg
	@${FIND} ${NEWBOOTDIR} -type f -perm +111 | ${XARGS} ${STRIP_CMD}
	(cd ${NEWBOOTDIR}/.. ; tar -cyf ${NEWBOOTNAME} bootstrap)


##
#
# The "cross-compiler" target will create a cross-compiler in the
# stage directory based on the value of ${CROSSTARG}.
# Execute it after the "patch" target
#
# The "cross-archive" will archive assembler code meant for linking
# on the native plaform.  It is run after "cross-compiler"
#
# The "native-link" target will bootstrap cm3 on a new platform (which
# obviously supports ports) using the archived files generated above.
# It could be done remotely via SSH, but this one target assumes files
# are locally available at ${WRKSRC}/${CROSSTARG}
#
##

CROSSTARG?=	AMD64_DRAGONFLY
CORELIBS=	m3-libs/m3core m3-libs/libm3
CORESYS=	m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker \
		m3-sys/m3back m3-sys/m3front m3-sys/m3quake m3-sys/cm3 \
		m3-tools/m3bundle
COREHELPERS=	m3-libs/sysutils m3-libs/patternmatching
COREFILES=	${CORELIBS} ${CORESYS}

cross-compiler:
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/build-cross-backend.sh ${CROSSTARG})
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \
		${SH} scripts/do-pkg.sh onlybuild \
		m3middle m3linker m3front m3quake cm3)
	@${FIND} ${WRKSRC} -name \.M3SHIP -print | ${XARGS} ${SED} -i -e \
		's|/bootstrap/|/stage${PREFIX}/|g'
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh ship \
		m3middle m3linker m3front m3quake cm3)	
	(cd ${WRKSRC}/m3-sys/cminstall/src/config-no-install && \
		${COPYTREE_SHARE} . ${STAGEDIR}${PREFIX}/etc/modula3)
	${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/cm3/${M3TARGET}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${M3TARGET}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${INSTALL_PROGRAM} ${WRKDIR}/bootstrap/bin/cm3cg \
		${STAGEDIR}${PREFIX}/bin/
	${INSTALL_PROGRAM} \
		${WRKSRC}/m3-sys/m3cc/${M3TARGET}-${CROSSTARG}/cm3cg \
		${STAGEDIR}${PREFIX}/bin/cm3cg-${CROSSTARG}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal \
		m3middle m3linker m3front m3quake cm3)
.for component in ${CORELIBS}
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T})
.endfor
	@(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${CORELIBS:T})
.for component in ${CORELIBS} ${COREHELPERS} ${CORESYS}
	@(cd ${WRKSRC}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh buildship ${component:T}; \
	    ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \
		${SH} scripts/do-pkg.sh cleanglobal ${component:T} \
	)
.endfor
	${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "m3back = INSTALL_ROOT & \"/bin/cm3cg-${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "BUILD_DIR = \"${CROSSTARG}\"" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "HOST = \"${M3TARGET}\"" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${CROSSTARG}\")" >> \
		${STAGEDIR}${PREFIX}/bin/cm3.cfg
	${ECHO} "M3_BOOTSTRAP = TRUE" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg
	@${ECHO} "====================================="
	@${ECHO} "=====  M3 cross-compiler ready  ====="
	@${ECHO} "====================================="
	@${ECHO}

assembler-code:
.for component in ${COREFILES}
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} SHIP=${TRUE} TARGET=${CROSSTARG} \
	    ${BUILTCM3} ${SH} scripts/do-pkg.sh onlybuild ${component:T} \
	)
.endfor
.for component in m3-libs/libm3 m3-sys/m3middle
	echo '#include "m3core.h"' > \
		${WRKSRC}/${component}/${CROSSTARG}/m3unix.h
.endfor
	${CP} ${WRKSRC}/m3-libs/m3core/src/m3core.h \
		${WRKSRC}/m3-libs/libm3/${CROSSTARG}/
	(cd ${WRKSRC}/m3-sys/m3middle; ${CP} src/POSIX/m3core.h ${CROSSTARG}/)

cross-archive: assembler-code
	(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} M3GDB=no ${BUILTCM3} \
		${SH} scripts/pack-crossbuild.sh ${CROSSTARG})

native-cm3cg:
# This target requires python to be installed
	@(cd ${WRKSRC}/m3-sys/m3cc; src/boot.py)
	@${MKDIR} ${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/m3cc/obj.1/gcc/m3cgc1 \
		${STAGEDIR}${PREFIX}/bin/cm3cg

native-link: native-cm3cg
.for component in ${COREFILES}
	@(cd ${WRKSRC}/${CROSSTARG}; ${TAR} -xzf ${component:T}.tgz)
.endfor
	@(cd ${WRKSRC}/scripts; \
		${SETENV} PATH=${PATH}:${STAGEDIR}${PREFIX}/bin \
		./boot-cm3-build-on-target.sh ${CROSSTARG})
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-sys/cm3/${CROSSTARG}/cm3 \
		${STAGEDIR}${PREFIX}/bin
	${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-tools/m3bundle/${CROSSTARG}/m3bundle \
		${STAGEDIR}${PREFIX}/bin

From jay.krell at cornell.edu  Sun Jan 19 13:17:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:17:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC081.1070401@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
Message-ID: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>

Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/0b2d21ba/attachment-0002.html>

From adacore at marino.st  Sun Jan 19 13:38:32 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 13:38:32 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <52DBC748.5090308@marino.st>

On 1/19/2014 13:17, Jay K wrote:
> Hey..I like that that does capture some of My Way. 
> Here is my independent off the cuff write up though:
>  I know the ports system is nice.
>  I know we really need to be in it for many operating systems.
>  I suspect your problem is that you didn't "upgrade" to apply you
> changes at the right time.

I must be misunderstanding what you mean by "upgrade".  It's the
cross-compiler that's complaining, and it was built by the latest patches.


>  Do any of these work for you:
>  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html

I'm not clear to which set you are referring.
There's no need for a binary set, modula3 now builds on FreeBSD 9 and
FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
superior to the release binaries because they link gmp and mpfr
statically while a bug in the cm3 sources caused those libraries to link
dynamically (which is why release binaries don't run on current FreeBSD)

Then there are the source distributions, but they are from 2012 so they
aren't exactly fresh either.


>  Anyway, please try this: 
>   Ignore DragonFly at first. 
>   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> anything. 
>   Such as from ports. 
>   Make a backup, e.g. of /usr/local/cm3. 
>   My instructions are unfortunately destructive of the existing install. 
>   Checkout the full current CVS repository. 
>   In that repository, run scripts/python/upgrade.py. 
>   If that doesn't work, stop, and we'll help. 

Is the goal just to get FreeBSD to have a binary with the latest
sources?  If so, I'd just modify the port to do that now that we have
working binary bootstraps.  The main reason I'm reluctant is that it
took over a week to make the cm3 port, I've never had so much difficulty
before.  (I've also spent another week on the DragonFly port).  So I'm
"resisting" because of how much work it already took.  That's why I was
trying to port DragonFly, THEN update to latest rather than update to
latest and then port DragonFly.  (in other words, going forward seems a
short path than backtracking)



>   If that does work, apply your diffs. And upgrade.py again. You can say
> "upgrade.py skipgcc" here. 
>   Then boot1.py c amd64_dragonflybsd   
>     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> on the command line. 
>     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> gcc/amd64 works for the same
>     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.

I assume you mean after the system understands this target.  There will
be a lot of files with __FreeBSD__ that need to be changed to
__FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
regenerate a bunch of patches from my current work.


>   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
>   Copy that to your Dragonfly system, extract it, cd into it, make. 
>   If that works, you have a native cm3. 
>   Run it. It should say "can't find cm3.cfg".  
>   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
>   cp cm3 /usr/local/cm3/bin  
>   export PATH=/usr/local/cm3/bin:$PATH
>   full CVS checkout  
>   cd scripts/python ; ./boot2.sh 
>    
>   You will need to have made small changes to pylib.py for this to work.
>   (And your config file should say to use the C backend, like Darwin and
> ARM_LINUX do.)


Well, I'm thrilled if I don't have to patch gcc again, that takes a long
time.  Shouldn't FreeBSD use the C backend too?

John


From jay.krell at cornell.edu  Sun Jan 19 13:32:44 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:32:44 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>, ,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>,
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>, ,
	<52DBC081.1070401@marino.st>,
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>
Message-ID: <COL130-W171FDF068BFCF6E20D357EE6BA0@phx.gbl>

Clarification:
upgrade.sh (note the .sh) rebuilds everything
I wrote upgrade.py based on it, but missed the "everything" part.
upgrade.py only upgrades "the compiler" (and m3core, libm3..)

A better practise although inefficient is:

 upgrade.py && ./do-cm3-all.py realclean skipgcc && ./do-cm3-all.py buildship  

I didn't suggest that to you here in order to get further in the pipeline faster.

However upgrade.py w/o other rebuild could leave m3core.so too new for many programs.
But for purposes of building/cross-building just the compiler, my instructions are good.


John, I see you added cm3 to the FreeBSD ports and you clearly know what you are doing.
Nevertheless, I am a lazy snob, don't quite see what you are doing wrong, and am suggesting "just different", not clearly better.
But, to my credit, this "just different" is what I use all the time.
I have used it to cross to several systems, w/ and w/o cm3cg (w/ and w/o C backend).
It isn't fast, and it doesn't always have the best incrementality, but it does usually work.
There is redundancy. pylib.py essentially duplicates parts of the config files.


You misunderstood what I meant by "C backend" and from the ports history you will somewhat appreciate it.
The C backend, instead of having a lot to do with gcc, is a backend that generates C, and then invokes a C compiler.
The problem you had with -gstabs for example, goes away. We'll use whatever debugging format regular C/C++ use.
There is a little hope of taking the one C output and compiling it for different targets -- at least FreeBSD 8,9,10.
Currently the C is very word size dependent, endian dependent, and somewhat otherwise target-dependent.
I'd like to fix these aspects but it isn't easy by far, will take possibly controversial changes in the frontend and interface to backend.


 - Jay




From: jay.krell at cornell.edu
To: adacore at marino.st
Date: Sun, 19 Jan 2014 12:17:24 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)




Hey..I like that that does capture some of My Way.


Here is my independent off the cuff write up though:


 I know the ports system is nice. 
 I know we really need to be in it for many operating systems. 


 I suspect your problem is that you didn't "upgrade" to apply you changes at the right time.

 Do any of these work for you: 


 https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html 


 Anyway, please try this: 


  Ignore DragonFly at first.  
  Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything.  
  Such as from ports.  
  Make a backup, e.g. of /usr/local/cm3.  
  My instructions are unfortunately destructive of the existing install.  
  Checkout the full current CVS repository.  
  In that repository, run scripts/python/upgrade.py.  
  If that doesn't work, stop, and we'll help.  
  If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here.  
  Then boot1.py c amd64_dragonflybsd    
    Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line.  
    Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same
    for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. 
  If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz  
  Copy that to your Dragonfly system, extract it, cd into it, make.  
  If that works, you have a native cm3.  
  Run it. It should say "can't find cm3.cfg".   
  Now on Dragonfly, mkdir -p /usr/local/cm3/bin  
  cp cm3 /usr/local/cm3/bin   
  export PATH=/usr/local/cm3/bin:$PATH 
  full CVS checkout   
  cd scripts/python ; ./boot2.sh  
    

  You will need to have made small changes to pylib.py for this to work. 
  (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.)




 - Jay




> Date: Sun, 19 Jan 2014 13:09:37 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 12:40, Jay K wrote:
> > Your change looks about right.
> >  
> > Did you first "upgrade" and existing install, with these changes, and
> > then cross?
> > That is what you must do.
> 
> 
> It's probably easier that I show you.
> I've attached "Makefile.local" (I added the .txt extension just now to
> help out some mail clients) which is my recipe.
> 
> So I run three targets in a freshly patched workarea:
> 1) make cross-compiler
> 2) make assembler-code (called from cross-archive)
> 3) make cross-archive
> 
> In the cross-compiler target, the FreeBSD bootstrap compiler builds
> m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler.
> 
> The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg
> Everything is cleaned, then the cross compiler builds everything again
> (including libm3core and libm3) along with m3objfile, sysutils, and
> patternmatching.
> 
> Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly,
> cm3cg is cm3cg-AMD64_DRAGONFLY.
> 
> I would not be surprised is there is a logic flaw or two in "make
> cross-compiler".  The "how to port CM3" tutorial was slightly out of
> date and some of the instructions were a bit ambiguous so this was a
> best guess.
> 
> Regards,
> John
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/153439e7/attachment-0002.html>

From jay.krell at cornell.edu  Sun Jan 19 13:50:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 12:50:27 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBC748.5090308@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
Message-ID: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>

I understand your reluctance.


> Is the goal just to get FreeBSD to have a binary with the latest
> sources?


Yes, and to make sure the build process works for you.


> Shouldn't FreeBSD use the C backend too?


Every target should. I haven't convinced everyone yet.
There is less resistance for new targets, like AMD64_NT and ARM_LINUX, which are working, using the C backend.
There are downsides to the C backend:
  - it is slower to compiler  
  - debugging is degraded on systems that have m3gdb  
 (debugging is much better on systems w/o m3gdb) 


And there are advantages not yet implemented, in particular, efficient exception handling by generating C++.


> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.


Your patches should apply fairly cleanly to any new checkout.
I am willing to through them and apply them to CVS if you want -- depends on if you are ok with my stealing your credit, vs. you want to commit them yourselves.
I already skimmed them and they all look easy.
DragonflyBSD is basically FreeBSD by another name.
Yes, a ton of work has been done on the kernel, file systems, maybe ports.
But the ABI is almost the same -- heck, for the most part, Linux==NetBSD==FreeBSD==OpenBSD.
Sure, they might be implemented differently, but they are highly source compatible and every highly object code compatible.


> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.

Agreed, we are both missing something simple.
Usually this is the error you get when you don't upgrade Target.i3/Target.m3.
If you want to press on with the current approach and ignore my advise, it should be possible to step through all this.
To start, break on Target__Init and see if it returns true or false.


> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)


Ugh. I saw something about that in your diffs. Definitely we intend to link statically.
I actually went to the trouble of removing use of gmp/mpfr/mpc in current source.
They are used for actually very little and aren't worth it.


 - Jay


> Date: Sun, 19 Jan 2014 13:38:32 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 13:17, Jay K wrote:
> > Hey..I like that that does capture some of My Way. 
> > Here is my independent off the cuff write up though:
> >  I know the ports system is nice.
> >  I know we really need to be in it for many operating systems.
> >  I suspect your problem is that you didn't "upgrade" to apply you
> > changes at the right time.
> 
> I must be misunderstanding what you mean by "upgrade".  It's the
> cross-compiler that's complaining, and it was built by the latest patches.
> 
> 
> >  Do any of these work for you:
> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
> 
> I'm not clear to which set you are referring.
> There's no need for a binary set, modula3 now builds on FreeBSD 9 and
> FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also
> superior to the release binaries because they link gmp and mpfr
> statically while a bug in the cm3 sources caused those libraries to link
> dynamically (which is why release binaries don't run on current FreeBSD)
> 
> Then there are the source distributions, but they are from 2012 so they
> aren't exactly fresh either.
> 
> 
> >  Anyway, please try this: 
> >   Ignore DragonFly at first. 
> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
> > anything. 
> >   Such as from ports. 
> >   Make a backup, e.g. of /usr/local/cm3. 
> >   My instructions are unfortunately destructive of the existing install. 
> >   Checkout the full current CVS repository. 
> >   In that repository, run scripts/python/upgrade.py. 
> >   If that doesn't work, stop, and we'll help. 
> 
> Is the goal just to get FreeBSD to have a binary with the latest
> sources?  If so, I'd just modify the port to do that now that we have
> working binary bootstraps.  The main reason I'm reluctant is that it
> took over a week to make the cm3 port, I've never had so much difficulty
> before.  (I've also spent another week on the DragonFly port).  So I'm
> "resisting" because of how much work it already took.  That's why I was
> trying to port DragonFly, THEN update to latest rather than update to
> latest and then port DragonFly.  (in other words, going forward seems a
> short path than backtracking)
> 
> 
> 
> >   If that does work, apply your diffs. And upgrade.py again. You can say
> > "upgrade.py skipgcc" here. 
> >   Then boot1.py c amd64_dragonflybsd   
> >     Make sure you say "c" and the target "amd64_dragonflybsd", anywhere
> > on the command line. 
> >     Ok, "c" is actually up to you. Since nobody messes with ABI much,
> > gcc/amd64 works for the same
> >     for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly.
> 
> I assume you mean after the system understands this target.  There will
> be a lot of files with __FreeBSD__ that need to be changed to
> __FreeBSD__ || __DragonFly__ as well.  In any case, I'll have to
> regenerate a bunch of patches from my current work.
> 
> 
> >   If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz 
> >   Copy that to your Dragonfly system, extract it, cd into it, make. 
> >   If that works, you have a native cm3. 
> >   Run it. It should say "can't find cm3.cfg".  
> >   Now on Dragonfly, mkdir -p /usr/local/cm3/bin 
> >   cp cm3 /usr/local/cm3/bin  
> >   export PATH=/usr/local/cm3/bin:$PATH
> >   full CVS checkout  
> >   cd scripts/python ; ./boot2.sh 
> >    
> >   You will need to have made small changes to pylib.py for this to work.
> >   (And your config file should say to use the C backend, like Darwin and
> > ARM_LINUX do.)
> 
> 
> Well, I'm thrilled if I don't have to patch gcc again, that takes a long
> time.  Shouldn't FreeBSD use the C backend too?
> 
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/ce74cb6a/attachment-0002.html>

From adacore at marino.st  Sun Jan 19 14:06:02 2014
From: adacore at marino.st (John Marino)
Date: Sun, 19 Jan 2014 14:06:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <52DBCDBA.8030809@marino.st>

On 1/19/2014 13:50, Jay K wrote:
> And there are advantages not yet implemented, in particular, efficient
> exception handling by generating C++.

Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
dl_iterate_phdr which gcc uses for efficient zero-cost exception
handling.  That might come in handy here (I implemented it in
DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)


> Your patches should apply fairly cleanly to any new checkout.
> I am willing to through them and apply them to CVS if you want --
> depends on if you are ok with my stealing your credit, vs. you want to
> commit them yourselves.

I would love it if you did this - I'm happy with a mention in the commit
message.  I just attached a tarball of all the patches I'm using right
now.  Some of these changes are specifically for the cross compiler (I'm
think of the /scripts changes mostly) but most are valid assuming they
still apply to the head of the repo.  Please take as much as you want
with my gratitude.

> I already skimmed them and they all look easy.
> DragonflyBSD is basically FreeBSD by another name.

That's not an accident.  As a DragonFly developer that mostly works on
userland, I've been converging FreeBSD and DragonFly again.  They
drifted apart after 10 years, but I thought it better to keep the
userlands as similar as possible for 3rd party applications.

> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> But the ABI is almost the same -- heck, for the most part,
> Linux==NetBSD==FreeBSD==OpenBSD.
> Sure, they might be implemented differently, but they are highly source
> compatible and every highly object code compatible.
>> I must be misunderstanding what you mean by "upgrade". It's the
>> cross-compiler that's complaining, and it was built by the latest patches.
> Agreed, we are both missing something simple.
> Usually this is the error you get when you don't upgrade
> Target.i3/Target.m3.
> If you want to press on with the current approach and ignore my advise,

I'm not ignoring it; it's more like I'm not quite ready to give up since
I'm so close.  I think I could actually get these libs and cm3 to build
on DragonFly if I used AMD64_FREEBSD as the target with the
cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
I'd then get stuck on the DragonFly side trying to compile cm3 with
itself, theoretically I should get the same error there.


> Ugh. I saw something about that in your diffs. Definitely we intend to
> link statically.
> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> current source.
> They are used for actually very little and aren't worth it.

Even better!
Regards,
John

-------------- next part --------------
A non-text attachment was scrubbed...
Name: files.tar
Type: application/x-tar
Size: 13062 bytes
Desc: not available
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/4ce4f917/attachment-0002.tar>

From dragisha at m3w.org  Sun Jan 19 15:28:18 2014
From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sun, 19 Jan 2014 15:28:18 +0100
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>
Message-ID: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>

Why would we do this, and not signalling, as in POSIX implementation?

Why try to make something synchronous when we have working asynchronous implementation?

Why would it be better than sync one?

On 19 Jan 2014, at 13:00, Jay K <jay.krell at cornell.edu> wrote:

> There is a list of threads maintained by the m3 runtime.
>   Not by enumerating OS threads.
> The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140119/be1f51b5/attachment-0002.sig>

From mika at async.caltech.edu  Sun Jan 19 21:05:48 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:05:48 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
Message-ID: <20140119200548.C18011A208F@async.async.caltech.edu>

"less worried" = "I think it works"?

I think the only threading that works in CM3 is user threading...

I tried to run CM3-compiled binaries through "valgrind" but got an
error that CM3 is using the same signal that valgrind does internally
(on AMD64_LINUX at least).  I think this is easy to change in m3core
(yes?) but haven't gotten around to it...

valgrind has a threading checker called "helgrind"...

     Mika

Jay writes:
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/plain;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>w up further.
>
> - Jay
>
>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>
>
>> Jay:
>> =20
>> I=E2=80=99m very concerned about the threading not working properly on bot=
>h 32-bit and 64-bit Windows.
>> =20
>> The Thread Test program crashes for me on both platforms.
>> =20
>> I haven=E2=80=99t tried your new test program yet.
>> =20
>> --Randy
>> =20
>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>> Sent: Saturday, January 18, 2014 1:23 AM
>> To: m3devel
>> Subject: EXT:[M3devel] Win32 SuspendThread?
>> =20
>>  This program also doesn't behave as expected, native, nothing to do with w=
>ow64.=20
>>  Anyone else please confirm:=20
>>    1) my expectations -- it should never print anything
>>    2) their importance  -- garbage collector depends on it =20
>>    3) their not being met -- stuff gets printed=20
>>  This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>  I am following up further.=20
>>  Maybe we should get cooperative suspend really going?=20
>> Thank you.
>>  - Jay=20
>> =20
>> #include <stdio.h>
>> #include <windows.h>
>> volatile long value;
>> unsigned long __stdcall Thread(PVOID parameter)
>> {
>>   while (1)
>>    InterlockedIncrement(&value);
>>   return 0;
>> }
>> int __cdecl main()
>> {
>>   HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>   UINT i =3D 0;
>>   while (1)
>>   {
>>     i +=3D 1;
>>     if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>     {
>>       printf("suspend failed %X\n", GetLastError());
>>       Sleep(1);
>>       continue;
>>     }
>>     volatile long a =3D value;=20
>>     volatile long b =3D value;
>>     if (a !=3D b)
>>     {
>>      printf("%d %d %d %d\n", i, a, b, b - a);
>>     }
>>     ResumeThread(thread);
>>   }
>> }
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>Content-Type: text/html;
>	charset=utf-8
>Content-Transfer-Encoding: quoted-printable
>
><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
><blockquote type=3D"cite"><div>
>
><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>
><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
><style><!--
>/* Font Definitions */
>@font-face
>	{font-family:Calibri;
>	panose-1:2 15 5 2 2 2 4 3 2 4;}
>@font-face
>	{font-family:Tahoma;
>	panose-1:2 11 6 4 3 5 4 4 2 4;}
>/* Style Definitions */
>p.MsoNormal, li.MsoNormal, div.MsoNormal
>	{margin:0in;
>	margin-bottom:.0001pt;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>a:link, span.MsoHyperlink
>	{mso-style-priority:99;
>	color:blue;
>	text-decoration:underline;}
>a:visited, span.MsoHyperlinkFollowed
>	{mso-style-priority:99;
>	color:purple;
>	text-decoration:underline;}
>p
>	{mso-style-priority:99;
>	mso-margin-top-alt:auto;
>	margin-right:0in;
>	mso-margin-bottom-alt:auto;
>	margin-left:0in;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>	{mso-style-priority:99;
>	mso-style-link:"Balloon Text Char";
>	margin:0in;
>	margin-bottom:.0001pt;
>	font-size:8.0pt;
>	font-family:"Tahoma","sans-serif";}
>span.EmailStyle18
>	{mso-style-type:personal-reply;
>	font-family:"Calibri","sans-serif";
>	color:#1F497D;}
>span.BalloonTextChar
>	{mso-style-name:"Balloon Text Char";
>	mso-style-priority:99;
>	mso-style-link:"Balloon Text";
>	font-family:"Tahoma","sans-serif";}
>.MsoChpDefault
>	{mso-style-type:export-only;
>	font-size:10.0pt;}
>@page WordSection1
>	{size:8.5in 11.0in;
>	margin:1.0in 1.0in 1.0in 1.0in;}
>div.WordSection1
>	{page:WordSection1;}
>--></style><!--[if gte mso 9]><xml>
><o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
></xml><![endif]--><!--[if gte mso 9]><xml>
><o:shapelayout v:ext=3D"edit">
><o:idmap v:ext=3D"edit" data=3D"1" />
></o:shapelayout></xml><![endif]-->
>
>
><div class=3D"WordSection1">
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>shes for me on both platforms.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>r new test program yet.<o:p></o:p></span></p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>p>
><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>
><div>
><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>in 0in">
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
><b>On Behalf Of </b>Jay K<br>
><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
><b>To:</b> m3devel<br>
><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
></div>
></div>
><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
><div>
><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>in-bottom:12.0pt;margin-left:.5in">
><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>;This program also doesn't behave as expected, native, nothing to do with wo=
>w64.
><br>
>&nbsp;Anyone else please confirm: <br>
>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>sp; <br>
>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>&nbsp;I am following up further. <br>
>&nbsp;Maybe we should get cooperative suspend really going? <br>
>Thank you.<br>
>&nbsp;- Jay <br>
>&nbsp;<br>
>#include &lt;stdio.h&gt;<br>
>#include &lt;windows.h&gt;<br>
>volatile long value;<br>
>unsigned long __stdcall Thread(PVOID parameter)<br>
>{<br>
>&nbsp; while (1)<br>
>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>&nbsp; return 0;<br>
>}<br>
>int __cdecl main()<br>
>{<br>
>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>&nbsp; UINT i =3D 0;<br>
>&nbsp; while (1)<br>
>&nbsp; {<br>
>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>;<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>&nbsp;&nbsp;&nbsp; {<br>
>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>&nbsp;&nbsp;&nbsp; }<br>
>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>&nbsp; }<br>
>}<o:p></o:p></span></p>
></div>
></div>
>
>
></div></blockquote></body></html>=
>
>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From mika at async.caltech.edu  Sun Jan 19 21:13:08 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 12:13:08 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
Message-ID: <20140119201308.0A4B41A208F@async.async.caltech.edu>


There were some bugs we never got around to addressing with the C backend,
weren't there?  A crash that didn't happen with native backend.

I've been using the "native" at more or less the head on very recent
Debian and the Raspberry Pi for a while now with no problems to speak of.
(As long as you only use user threads... but that's another story, I think.)

     Mika

Jay K writes:
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>I understand your reluctance.
>
>
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?
>
>
>Yes=2C and to make sure the build process works for you.
>
>
>> Shouldn't FreeBSD use the C backend too?
>
>
>Every target should. I haven't convinced everyone yet.
>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>which are working=2C using the C backend.
>There are downsides to the C backend:
>  - it is slower to compiler =20
>  - debugging is degraded on systems that have m3gdb =20
> (debugging is much better on systems w/o m3gdb)=20
>
>
>And there are advantages not yet implemented=2C in particular=2C efficient =
>exception handling by generating C++.
>
>
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>
>
>Your patches should apply fairly cleanly to any new checkout.
>I am willing to through them and apply them to CVS if you want -- depends o=
>n if you are ok with my stealing your credit=2C vs. you want to commit them=
> yourselves.
>I already skimmed them and they all look easy.
>DragonflyBSD is basically FreeBSD by another name.
>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>orts.
>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>Sure=2C they might be implemented differently=2C but they are highly source=
> compatible and every highly object code compatible.
>
>
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>
>Agreed=2C we are both missing something simple.
>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>3.
>If you want to press on with the current approach and ignore my advise=2C i=
>t should be possible to step through all this.
>To start=2C break on Target__Init and see if it returns true or false.
>
>
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>
>
>Ugh. I saw something about that in your diffs. Definitely we intend to link=
> statically.
>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>ource.
>They are used for actually very little and aren't worth it.
>
>
> - Jay
>
>
>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>> From: adacore at marino.st
>> To: jay.krell at cornell.edu
>> CC: m3devel at elegosoft.com
>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>=20
>> On 1/19/2014 13:17=2C Jay K wrote:
>> > Hey..I like that that does capture some of My Way.=20
>> > Here is my independent off the cuff write up though:
>> >  I know the ports system is nice.
>> >  I know we really need to be in it for many operating systems.
>> >  I suspect your problem is that you didn't "upgrade" to apply you
>> > changes at the right time.
>>=20
>> I must be misunderstanding what you mean by "upgrade".  It's the
>> cross-compiler that's complaining=2C and it was built by the latest patch=
>es.
>>=20
>>=20
>> >  Do any of these work for you:
>> >  https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>=20
>> I'm not clear to which set you are referring.
>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>> superior to the release binaries because they link gmp and mpfr
>> statically while a bug in the cm3 sources caused those libraries to link
>> dynamically (which is why release binaries don't run on current FreeBSD)
>>=20
>> Then there are the source distributions=2C but they are from 2012 so they
>> aren't exactly fresh either.
>>=20
>>=20
>> >  Anyway=2C please try this:=20
>> >   Ignore DragonFly at first.=20
>> >   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>> > anything.=20
>> >   Such as from ports.=20
>> >   Make a backup=2C e.g. of /usr/local/cm3.=20
>> >   My instructions are unfortunately destructive of the existing install=
>.=20
>> >   Checkout the full current CVS repository.=20
>> >   In that repository=2C run scripts/python/upgrade.py.=20
>> >   If that doesn't work=2C stop=2C and we'll help.=20
>>=20
>> Is the goal just to get FreeBSD to have a binary with the latest
>> sources?  If so=2C I'd just modify the port to do that now that we have
>> working binary bootstraps.  The main reason I'm reluctant is that it
>> took over a week to make the cm3 port=2C I've never had so much difficult=
>y
>> before.  (I've also spent another week on the DragonFly port).  So I'm
>> "resisting" because of how much work it already took.  That's why I was
>> trying to port DragonFly=2C THEN update to latest rather than update to
>> latest and then port DragonFly.  (in other words=2C going forward seems a
>> short path than backtracking)
>>=20
>>=20
>>=20
>> >   If that does work=2C apply your diffs. And upgrade.py again. You can =
>say
>> > "upgrade.py skipgcc" here.=20
>> >   Then boot1.py c amd64_dragonflybsd  =20
>> >     Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>re
>> > on the command line.=20
>> >     Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>=2C
>> > gcc/amd64 works for the same
>> >     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>=20
>> I assume you mean after the system understands this target.  There will
>> be a lot of files with __FreeBSD__ that need to be changed to
>> __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to
>> regenerate a bunch of patches from my current work.
>>=20
>>=20
>> >   If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>gz=20
>> >   Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>e.=20
>> >   If that works=2C you have a native cm3.=20
>> >   Run it. It should say "can't find cm3.cfg". =20
>> >   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>> >   cp cm3 /usr/local/cm3/bin =20
>> >   export PATH=3D/usr/local/cm3/bin:$PATH
>> >   full CVS checkout =20
>> >   cd scripts/python =3B ./boot2.sh=20
>> >   =20
>> >   You will need to have made small changes to pylib.py for this to work=
>.
>> >   (And your config file should say to use the C backend=2C like Darwin =
>and
>> > ARM_LINUX do.)
>>=20
>>=20
>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>ong
>> time.  Shouldn't FreeBSD use the C backend too?
>>=20
>> John
> 		 	   		  =
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 12pt=3B
>font-family:Calibri
>}
>--></style></head>
><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>h are working=2C using the C backend.<br>There are downsides to the C backe=
>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>et implemented=2C in particular=2C efficient exception handling by generati=
>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>=3B     for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>=3B <br>&gt=3B I assume you mean after the system understands this target. =
> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well.  In any case=2C I'=
>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
> am willing to through them and apply them to CVS if you want -- depends on=
> if you are ok with my stealing your credit=2C vs. you want to commit them =
>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>e highly source compatible and every highly object code compatible.<br><br>=
><br>&gt=3B I must be misunderstanding what you mean by "upgrade".  It's the=
><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>r>If you want to press on with the current approach and ignore my advise=2C=
> it should be possible to step through all this.<br>To start=2C break on Ta=
>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
> to link statically.<br>I actually went to the trouble of removing use of g=
>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>dependent off the cuff write up though:<br>&gt=3B &gt=3B  I know the ports =
>system is nice.<br>&gt=3B &gt=3B  I know we really need to be in it for man=
>y operating systems.<br>&gt=3B &gt=3B  I suspect your problem is that you d=
>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade".  =
>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Do any of thes=
>e work for you:<br>&gt=3B &gt=3B  https://modula3.elegosoft.com/cm3/snaps/s=
>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B  Anyway=2C plea=
>se try this: <br>&gt=3B &gt=3B   Ignore DragonFly at first. <br>&gt=3B &gt=
>=3B   Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B   Such as from ports. <br>&gt=3B =
>&gt=3B   Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B   My in=
>structions are unfortunately destructive of the existing install. <br>&gt=
>=3B &gt=3B   Checkout the full current CVS repository. <br>&gt=3B &gt=3B   =
>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B   If=
> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>?  If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>working binary bootstraps.  The main reason I'm reluctant is that it<br>&gt=
>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>lty<br>&gt=3B before.  (I've also spent another week on the DragonFly port)=
>.  So I'm<br>&gt=3B "resisting" because of how much work it already took.  =
>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>t rather than update to<br>&gt=3B latest and then port DragonFly.  (in othe=
>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B   If that does work=2C appl=
>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>y skipgcc" here. <br>&gt=3B &gt=3B   Then boot1.py c amd64_dragonflybsd   <=
>br>&gt=3B &gt=3B     Make sure you say "c" and the target "amd64_dragonflyb=
>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B    =
> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B     for Linux=2C=
> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>me you mean after the system understands this target.  There will<br>&gt=3B=
> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>FreeBSD__ || __DragonFly__ as well.  In any case=2C I'll have to<br>&gt=3B =
>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>r>&gt=3B &gt=3B   If this works=2C it will give you=2C roughly amd64_dragon=
>flybsd*.tar.gz <br>&gt=3B &gt=3B   Copy that to your Dragonfly system=2C ex=
>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B   If that works=2C you ha=
>ve a native cm3. <br>&gt=3B &gt=3B   Run it. It should say "can't find cm3.=
>cfg".  <br>&gt=3B &gt=3B   Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
><br>&gt=3B &gt=3B   cp cm3 /usr/local/cm3/bin  <br>&gt=3B &gt=3B   export P=
>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B   full CVS checkout  <br>&g=
>t=3B &gt=3B   cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B    <br>&gt=
>=3B &gt=3B   You will need to have made small changes to pylib.py for this =
>to work.<br>&gt=3B &gt=3B   (And your config file should say to use the C b=
>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>hat takes a long<br>&gt=3B time.  Shouldn't FreeBSD use the C backend too?<=
>br>&gt=3B <br>&gt=3B John<br></div> 		 	   		  </div></body>
></html>=
>
>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_--


From mika at async.caltech.edu  Sun Jan 19 22:36:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 13:36:49 -0800
Subject: [M3devel] narrow still failing..
Message-ID: <20140119213649.403AE1A208F@async.async.caltech.edu>


A smaller test:

     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  PROCEDURE Z(VAR x : REFANY) =
    20    BEGIN
    21      x := BDD.Format(x)
    22    END Z;
    23
    24  BEGIN
    25    Q()
    26  END Main.

(191)rover:~/refany/src>../FreeBSD4/refany

***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 21
***

What's going on here???

Here are the relevant declarations:

(* simple BDD package *)

INTERFACE BDD;
IMPORT Word;

TYPE 
  T <: REFANY;

(* the boolean constants top and bottom *)
PROCEDURE True() : T;
PROCEDURE False() : T;

(* a new variable *)
PROCEDURE New(name : TEXT := NIL) : T;

(* unary ops *)
PROCEDURE Not(a : T) : T;

(* binary ops *)
PROCEDURE And(a, b : T) : T;
PROCEDURE Xor(a , b : T) : T;
PROCEDURE Equivalent(a, b : T) : T;
PROCEDURE Or( a, b : T) : T;
PROCEDURE Implies  (a , b : T) : T;

(* maketrue/makefalse *)

PROCEDURE MakeFalse(b, v : T) : T; (* make v false in b *)
PROCEDURE MakeTrue(b, v : T) : T; (* make v true in b *)


(* print with ids *)
PROCEDURE Format(a : T; symtab : REFANY (* BDDTextTbl.T *) := NIL) : TEXT;

(* the following procedures allow this interface to be used in generics *)
PROCEDURE Equal(a, b : T) : BOOLEAN;
PROCEDURE Hash(a : T) : Word.T;

PROCEDURE Size(a : T) : CARDINAL;
  (* number of nodes in structure *)

CONST
  Brand = "BDD 0.1";

PROCEDURE GetId(a : T) : INTEGER; (* for debugging *)

END BDD.
....

REVEAL
  T = BRANDED Brand OBJECT
    l , r : T;
    root : Root;
    tag : CARDINAL; (* for hashing *)
    name : TEXT;
  METHODS
    init() : T := Init;
  END;

PROCEDURE New(name : TEXT) : T =
  VAR res : Root := NEW(Root).init(); BEGIN
    res.name := name;
    res.root := res;
    res.l := true; res.r := false;
    res.id := nextId;
    res.tab := NEW(BDDTripleHash.Default).init(128);
    FOR i := FIRST(res.cache) TO LAST(res.cache) DO
      res.cache[i] := NEW(BDDTripleHash.Default).init(64)
    END; 
    EVAL BDDTripleHash.Put(res.tab, Pair { res.l, res.r }, (res));
    INC(nextId);
    RETURN res
  END New;

PROCEDURE Format(x : T; symtab : REFANY := NIL) : TEXT =
  VAR nm : TEXT;
  BEGIN
    IF symtab # NIL AND NARROW(symtab, BDDTextTbl.T).get(x, nm) THEN
      RETURN nm
    END;
    IF x = true THEN RETURN "TRUE"
    ELSIF x = false THEN RETURN "FALSE"
    END;
       
    IF x.name # NIL THEN RETURN x.name END;

    RETURN Fmt.Int(x.root.id) & " && (" & Format(x.l) & ") || (" & Format(x.r) &
           ") && ~" & Fmt.Int(x.root.id)
  END Format;

  use option @M3stackdump to get a stack trace
Abort


I'm happy to share the whole package if anyone with a better clue than me would like a look.

In fact:

http://rover.gcapltd.com/~mika/bdd.tgz

It should build without anything outside of standard cm3. 

     Mika


From mika at async.caltech.edu  Sun Jan 19 23:01:23 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 14:01:23 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119213649.403AE1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
Message-ID: <20140119220123.0D1971A208F@async.async.caltech.edu>

As in the ISTYPE/TYPECASE example, the following works:


PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(b)
    END;
  END Z;

(244)rover:~/refany/src>../FreeBSD4/refany
a


This does NOT work:

PROCEDURE Z(VAR x : REFANY) =
  BEGIN
    TYPECASE x OF
      BDD.T(b) => x := BDD.Format(x)
    END;
  END Z;


(242)rover:~/refany/src>../FreeBSD4/refany


***
*** runtime error:
***    NARROW failed
***    file "/big/home/mika/refany/src/Main.m3", line 22
***

  use option @M3stackdump to get a stack trace
Abort


Same result PM3 / CM3.

     Mika


From jay.krell at cornell.edu  Sun Jan 19 23:58:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 22:58:27 +0000
Subject: [M3devel] Cooperative suspend?
In-Reply-To: <DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org>
	<COL130-W729C4C8F6673A0FC8BFD6BE6BA0@phx.gbl>,
	<DBDB6492-2537-4797-9D9B-28646C72EE8D@m3w.org>
Message-ID: <COL130-W23436BD09BE1F20C442D6FE6BA0@phx.gbl>

1) portability; We currently have separate code, roughly, for MacOSX, FreeBSD, OpenBSD, NT, and the other Posixes.
MacOSX doesn't have Posix semaphores, for example.
This would give us one codebase here.
Only IA64 would likely remain an outlier, because stack is not merely the address of one local to another, but includes also the grow-up stack for the register saves.



2) It'd workaround the wow64 problem.


?- Jay

________________________________
> Subject: Re: [M3devel] Cooperative suspend? 
> From: dragisha at m3w.org 
> Date: Sun, 19 Jan 2014 15:28:18 +0100 
> CC: m3devel at elegosoft.com 
> To: jay.krell at cornell.edu 
> 
> Why would we do this, and not signalling, as in POSIX implementation? 
> 
> Why try to make something synchronous when we have working asynchronous 
> implementation? 
> 
> Why would it be better than sync one? 
> 
> On 19 Jan 2014, at 13:00, Jay K 
> <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote: 
> 
> There is a list of threads maintained by the m3 runtime. 
> Not by enumerating OS threads. 
> The thread structure contains in it perhaps a boolean or enum as to 
> suspend requested, suspending, suspended, resuming, running, etc. 
> 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:13:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:13:24 +0000
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>,
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <COL130-W156EFA4699196726508DB4E6BA0@phx.gbl>

"not worried" for native windows.
Still worried about wow64.


The answer for native is in Rotor/sscli and I have a program to show it working/not-working.
Specifically SuspendThread needs to be followed by GetThreadContext to ensure the suspend has completed.
If we don't do that, we must. If we already do, we are ok. I'll look later.
wow64 is still a problem. I'm still looking into it and thinking about it. But we do have native AMD64_NT
so just saying wow64 doesn't work might be the answer. I know people will protest that it works, but doesn't always.
I haven't figured out what rotor/sscli does.
Java uses cooperative suspend, which sidesteps this stuff.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Win32 SuspendThread?
> Date: Sun, 19 Jan 2014 12:05:48 -0800
> From: mika at async.caltech.edu
>
> "less worried" = "I think it works"?
>
> I think the only threading that works in CM3 is user threading...
>
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least). I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
>
> valgrind has a threading checker called "helgrind"...
>
> Mika
>
> Jay writes:
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/plain;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>>For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>>w up further.
>>
>> - Jay
>>
>>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>>
>>
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>>h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>>ow64.=20
>>> Anyone else please confirm:=20
>>> 1) my expectations -- it should never print anything
>>> 2) their importance -- garbage collector depends on it =20
>>> 3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>> while (1)
>>> InterlockedIncrement(&value);
>>> return 0;
>>> }
>>> int __cdecl main()
>>> {
>>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>> UINT i =3D 0;
>>> while (1)
>>> {
>>> i +=3D 1;
>>> if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>> {
>>> printf("suspend failed %X\n", GetLastError());
>>> Sleep(1);
>>> continue;
>>> }
>>> volatile long a =3D value;=20
>>> volatile long b =3D value;
>>> if (a !=3D b)
>>> {
>>> printf("%d %d %d %d\n", i, a, b, b - a);
>>> }
>>> ResumeThread(thread);
>>> }
>>> }
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>>Content-Type: text/html;
>> charset=utf-8
>>Content-Transfer-Encoding: quoted-printable
>>
>><html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>>utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>>ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>>div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>><blockquote type=3D"cite"><div>
>>
>><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>>
>><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>><style>
>>
>>
>><div class=3D"WordSection1">
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>>bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>>:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>>shes for me on both platforms.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>>r new test program yet.<o:p></o:p></span></p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>>p>
>><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>>ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>>>
>><div>
>><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>>in 0in">
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>>e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>></b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>>s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>>a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>><b>On Behalf Of </b>Jay K<br>
>><b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>><b>To:</b> m3devel<br>
>><b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>></div>
>></div>
>><p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>><div>
>><p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>>in-bottom:12.0pt;margin-left:.5in">
>><span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>>;This program also doesn't behave as expected, native, nothing to do with wo=
>>w64.
>><br>
>>&nbsp;Anyone else please confirm: <br>
>>&nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>>&nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>>sp; <br>
>>&nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>>&nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>>&nbsp;I am following up further. <br>
>>&nbsp;Maybe we should get cooperative suspend really going? <br>
>>Thank you.<br>
>>&nbsp;- Jay <br>
>>&nbsp;<br>
>>#include &lt;stdio.h&gt;<br>
>>#include &lt;windows.h&gt;<br>
>>volatile long value;<br>
>>unsigned long __stdcall Thread(PVOID parameter)<br>
>>{<br>
>>&nbsp; while (1)<br>
>>&nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>>&nbsp; return 0;<br>
>>}<br>
>>int __cdecl main()<br>
>>{<br>
>>&nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>>&nbsp; UINT i =3D 0;<br>
>>&nbsp; while (1)<br>
>>&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>>&nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>>;<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>>&nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>>&nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>>&nbsp;&nbsp;&nbsp; {<br>
>>&nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>>&nbsp;&nbsp;&nbsp; }<br>
>>&nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>>&nbsp; }<br>
>>}<o:p></o:p></span></p>
>></div>
>></div>
>>
>>
>></div></blockquote></body></html>=
>>
>>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- 		 	   		  

From jay.krell at cornell.edu  Mon Jan 20 00:15:02 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 19 Jan 2014 23:15:02 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <20140119201308.0A4B41A208F@async.async.caltech.edu>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
Message-ID: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>

The only problem I remember is an alignment matter using msvcrt.dll on AMD64_NT.
I'll look into it, and the old emails.
I've used the C backend on many platforms now.


?- Jay

----------------------------------------
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com; adacore at marino.st
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> Date: Sun, 19 Jan 2014 12:13:08 -0800
> From: mika at async.caltech.edu
>
>
> There were some bugs we never got around to addressing with the C backend,
> weren't there? A crash that didn't happen with native backend.
>
> I've been using the "native" at more or less the head on very recent
> Debian and the Raspberry Pi for a while now with no problems to speak of.
> (As long as you only use user threads... but that's another story, I think.)
>
> Mika
>
> Jay K writes:
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/plain; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>>I understand your reluctance.
>>
>>
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources?
>>
>>
>>Yes=2C and to make sure the build process works for you.
>>
>>
>>> Shouldn't FreeBSD use the C backend too?
>>
>>
>>Every target should. I haven't convinced everyone yet.
>>There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C =
>>which are working=2C using the C backend.
>>There are downsides to the C backend:
>> - it is slower to compiler =20
>> - debugging is degraded on systems that have m3gdb =20
>> (debugging is much better on systems w/o m3gdb)=20
>>
>>
>>And there are advantages not yet implemented=2C in particular=2C efficient =
>>exception handling by generating C++.
>>
>>
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>
>>
>>Your patches should apply fairly cleanly to any new checkout.
>>I am willing to through them and apply them to CVS if you want -- depends o=
>>n if you are ok with my stealing your credit=2C vs. you want to commit them=
>> yourselves.
>>I already skimmed them and they all look easy.
>>DragonflyBSD is basically FreeBSD by another name.
>>Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p=
>>orts.
>>But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN=
>>etBSD=3D=3DFreeBSD=3D=3DOpenBSD.
>>Sure=2C they might be implemented differently=2C but they are highly source=
>> compatible and every highly object code compatible.
>>
>>
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>
>>Agreed=2C we are both missing something simple.
>>Usually this is the error you get when you don't upgrade Target.i3/Target.m=
>>3.
>>If you want to press on with the current approach and ignore my advise=2C i=
>>t should be possible to step through all this.
>>To start=2C break on Target__Init and see if it returns true or false.
>>
>>
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>
>>
>>Ugh. I saw something about that in your diffs. Definitely we intend to link=
>> statically.
>>I actually went to the trouble of removing use of gmp/mpfr/mpc in current s=
>>ource.
>>They are used for actually very little and aren't worth it.
>>
>>
>> - Jay
>>
>>
>>> Date: Sun=2C 19 Jan 2014 13:38:32 +0100
>>> From: adacore at marino.st
>>> To: jay.krell at cornell.edu
>>> CC: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
>>>=20
>>> On 1/19/2014 13:17=2C Jay K wrote:
>>>> Hey..I like that that does capture some of My Way.=20
>>>> Here is my independent off the cuff write up though:
>>>> I know the ports system is nice.
>>>> I know we really need to be in it for many operating systems.
>>>> I suspect your problem is that you didn't "upgrade" to apply you
>>>> changes at the right time.
>>>=20
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining=2C and it was built by the latest patch=
>>es.
>>>=20
>>>=20
>>>> Do any of these work for you:
>>>> https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html
>>>=20
>>> I'm not clear to which set you are referring.
>>> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and
>>> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also
>>> superior to the release binaries because they link gmp and mpfr
>>> statically while a bug in the cm3 sources caused those libraries to link
>>> dynamically (which is why release binaries don't run on current FreeBSD)
>>>=20
>>> Then there are the source distributions=2C but they are from 2012 so they
>>> aren't exactly fresh either.
>>>=20
>>>=20
>>>> Anyway=2C please try this:=20
>>>> Ignore DragonFly at first.=20
>>>> Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
>>>> anything.=20
>>>> Such as from ports.=20
>>>> Make a backup=2C e.g. of /usr/local/cm3.=20
>>>> My instructions are unfortunately destructive of the existing install=
>>.=20
>>>> Checkout the full current CVS repository.=20
>>>> In that repository=2C run scripts/python/upgrade.py.=20
>>>> If that doesn't work=2C stop=2C and we'll help.=20
>>>=20
>>> Is the goal just to get FreeBSD to have a binary with the latest
>>> sources? If so=2C I'd just modify the port to do that now that we have
>>> working binary bootstraps. The main reason I'm reluctant is that it
>>> took over a week to make the cm3 port=2C I've never had so much difficult=
>>y
>>> before. (I've also spent another week on the DragonFly port). So I'm
>>> "resisting" because of how much work it already took. That's why I was
>>> trying to port DragonFly=2C THEN update to latest rather than update to
>>> latest and then port DragonFly. (in other words=2C going forward seems a
>>> short path than backtracking)
>>>=20
>>>=20
>>>=20
>>>> If that does work=2C apply your diffs. And upgrade.py again. You can =
>>say
>>>> "upgrade.py skipgcc" here.=20
>>>> Then boot1.py c amd64_dragonflybsd =20
>>>> Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe=
>>re
>>>> on the command line.=20
>>>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=
>>=2C
>>>> gcc/amd64 works for the same
>>>> for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>>>=20
>>> I assume you mean after the system understands this target. There will
>>> be a lot of files with __FreeBSD__ that need to be changed to
>>> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>>> regenerate a bunch of patches from my current work.
>>>=20
>>>=20
>>>> If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.=
>>gz=20
>>>> Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak=
>>e.=20
>>>> If that works=2C you have a native cm3.=20
>>>> Run it. It should say "can't find cm3.cfg". =20
>>>> Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20
>>>> cp cm3 /usr/local/cm3/bin =20
>>>> export PATH=3D/usr/local/cm3/bin:$PATH
>>>> full CVS checkout =20
>>>> cd scripts/python =3B ./boot2.sh=20
>>>> =20
>>>> You will need to have made small changes to pylib.py for this to work=
>>.
>>>> (And your config file should say to use the C backend=2C like Darwin =
>>and
>>>> ARM_LINUX do.)
>>>=20
>>>=20
>>> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l=
>>ong
>>> time. Shouldn't FreeBSD use the C backend too?
>>>=20
>>> John
>> =
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_
>>Content-Type: text/html; charset="iso-8859-1"
>>Content-Transfer-Encoding: quoted-printable
>>
>><html>
>><head>
>><style></style></head>
>><body class=3D'hmmessage'><div dir=3D'ltr'>I understand your reluctance.<br=
>>><br><br>&gt=3B Is the goal just to get FreeBSD to have a binary with the l=
>>atest<br>&gt=3B sources?<br><br><br>Yes=2C and to make sure the build proce=
>>ss works for you.<br><br><br>&gt=3B Shouldn't FreeBSD use the C backend too=
>>?<br><br><br>Every target should. I haven't convinced everyone yet.<br>Ther=
>>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic=
>>h are working=2C using the C backend.<br>There are downsides to the C backe=
>>nd:<br>&nbsp=3B - it is slower to compiler&nbsp=3B <br>&nbsp=3B - debugging=
>> is degraded on systems that have m3gdb&nbsp=3B <br>&nbsp=3B(debugging is m=
>>uch better on systems w/o m3gdb) <br><br><br>And there are advantages not y=
>>et implemented=2C in particular=2C efficient exception handling by generati=
>>ng C++.<br><br><br>&gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=
>>=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=
>>=3B <br>&gt=3B I assume you mean after the system understands this target. =
>> There will<br>&gt=3B be a lot of files with __FreeBSD__ that need to be ch=
>>anged to<br>&gt=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'=
>>ll have to<br>&gt=3B regenerate a bunch of patches from my current work.<br=
>>><br><br>Your patches should apply fairly cleanly to any new checkout.<br>I=
>> am willing to through them and apply them to CVS if you want -- depends on=
>> if you are ok with my stealing your credit=2C vs. you want to commit them =
>>yourselves.<br>I already skimmed them and they all look easy.<br>DragonflyB=
>>SD is basically FreeBSD by another name.<br>Yes=2C a ton of work has been d=
>>one on the kernel=2C file systems=2C maybe ports.<br>But the ABI is almost =
>>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D=
>>=3DOpenBSD.<br>Sure=2C they might be implemented differently=2C but they ar=
>>e highly source compatible and every highly object code compatible.<br><br>=
>><br>&gt=3B I must be misunderstanding what you mean by "upgrade". It's the=
>><br>&gt=3B cross-compiler that's complaining=2C and it was built by the lat=
>>est patches.<br><br>Agreed=2C we are both missing something simple.<br>Usua=
>>lly this is the error you get when you don't upgrade Target.i3/Target.m3.<b=
>>r>If you want to press on with the current approach and ignore my advise=2C=
>> it should be possible to step through all this.<br>To start=2C break on Ta=
>>rget__Init and see if it returns true or false.<br><br><br>&gt=3B staticall=
>>y while a bug in the cm3 sources caused those libraries to link<br>&gt=3B d=
>>ynamically (which is why release binaries don't run on current FreeBSD)<br>=
>><br><br>Ugh. I saw something about that in your diffs. Definitely we intend=
>> to link statically.<br>I actually went to the trouble of removing use of g=
>>mp/mpfr/mpc in current source.<br>They are used for actually very little an=
>>d aren't worth it.<br><br><br>&nbsp=3B- Jay<br><br><br><div>&gt=3B Date: Su=
>>n=2C 19 Jan 2014 13:38:32 +0100<br>&gt=3B From: adacore at marino.st<br>&gt=3B=
>> To: jay.krell at cornell.edu<br>&gt=3B CC: m3devel at elegosoft.com<br>&gt=3B Su=
>>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)<br>&g=
>>t=3B <br>&gt=3B On 1/19/2014 13:17=2C Jay K wrote:<br>&gt=3B &gt=3B Hey..I =
>>like that that does capture some of My Way. <br>&gt=3B &gt=3B Here is my in=
>>dependent off the cuff write up though:<br>&gt=3B &gt=3B I know the ports =
>>system is nice.<br>&gt=3B &gt=3B I know we really need to be in it for man=
>>y operating systems.<br>&gt=3B &gt=3B I suspect your problem is that you d=
>>idn't "upgrade" to apply you<br>&gt=3B &gt=3B changes at the right time.<br=
>>>&gt=3B <br>&gt=3B I must be misunderstanding what you mean by "upgrade". =
>>It's the<br>&gt=3B cross-compiler that's complaining=2C and it was built by=
>> the latest patches.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Do any of thes=
>>e work for you:<br>&gt=3B &gt=3B https://modula3.elegosoft.com/cm3/snaps/s=
>>napshot-index.html<br>&gt=3B <br>&gt=3B I'm not clear to which set you are =
>>referring.<br>&gt=3B There's no need for a binary set=2C modula3 now builds=
>> on FreeBSD 9 and<br>&gt=3B FreeBSD 10 (http://www.freshports.org/lang/modu=
>>la3)=2C which is also<br>&gt=3B superior to the release binaries because th=
>>ey link gmp and mpfr<br>&gt=3B statically while a bug in the cm3 sources ca=
>>used those libraries to link<br>&gt=3B dynamically (which is why release bi=
>>naries don't run on current FreeBSD)<br>&gt=3B <br>&gt=3B Then there are th=
>>e source distributions=2C but they are from 2012 so they<br>&gt=3B aren't e=
>>xactly fresh either.<br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B Anyway=2C plea=
>>se try this: <br>&gt=3B &gt=3B Ignore DragonFly at first. <br>&gt=3B &gt=
>>=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost<br>=
>>&gt=3B &gt=3B anything. <br>&gt=3B &gt=3B Such as from ports. <br>&gt=3B =
>>&gt=3B Make a backup=2C e.g. of /usr/local/cm3. <br>&gt=3B &gt=3B My in=
>>structions are unfortunately destructive of the existing install. <br>&gt=
>>=3B &gt=3B Checkout the full current CVS repository. <br>&gt=3B &gt=3B =
>>In that repository=2C run scripts/python/upgrade.py. <br>&gt=3B &gt=3B If=
>> that doesn't work=2C stop=2C and we'll help. <br>&gt=3B <br>&gt=3B Is the =
>>goal just to get FreeBSD to have a binary with the latest<br>&gt=3B sources=
>>? If so=2C I'd just modify the port to do that now that we have<br>&gt=3B =
>>working binary bootstraps. The main reason I'm reluctant is that it<br>&gt=
>>=3B took over a week to make the cm3 port=2C I've never had so much difficu=
>>lty<br>&gt=3B before. (I've also spent another week on the DragonFly port)=
>>. So I'm<br>&gt=3B "resisting" because of how much work it already took. =
>>That's why I was<br>&gt=3B trying to port DragonFly=2C THEN update to lates=
>>t rather than update to<br>&gt=3B latest and then port DragonFly. (in othe=
>>r words=2C going forward seems a<br>&gt=3B short path than backtracking)<br=
>>>&gt=3B <br>&gt=3B <br>&gt=3B <br>&gt=3B &gt=3B If that does work=2C appl=
>>y your diffs. And upgrade.py again. You can say<br>&gt=3B &gt=3B "upgrade.p=
>>y skipgcc" here. <br>&gt=3B &gt=3B Then boot1.py c amd64_dragonflybsd <=
>>br>&gt=3B &gt=3B Make sure you say "c" and the target "amd64_dragonflyb=
>>sd"=2C anywhere<br>&gt=3B &gt=3B on the command line. <br>&gt=3B &gt=3B =
>> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C<br>&=
>>gt=3B &gt=3B gcc/amd64 works for the same<br>&gt=3B &gt=3B for Linux=2C=
>> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.<br>&gt=3B <br>&gt=3B I assu=
>>me you mean after the system understands this target. There will<br>&gt=3B=
>> be a lot of files with __FreeBSD__ that need to be changed to<br>&gt=3B __=
>>FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to<br>&gt=3B =
>>regenerate a bunch of patches from my current work.<br>&gt=3B <br>&gt=3B <b=
>>r>&gt=3B &gt=3B If this works=2C it will give you=2C roughly amd64_dragon=
>>flybsd*.tar.gz <br>&gt=3B &gt=3B Copy that to your Dragonfly system=2C ex=
>>tract it=2C cd into it=2C make. <br>&gt=3B &gt=3B If that works=2C you ha=
>>ve a native cm3. <br>&gt=3B &gt=3B Run it. It should say "can't find cm3.=
>>cfg". <br>&gt=3B &gt=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin =
>><br>&gt=3B &gt=3B cp cm3 /usr/local/cm3/bin <br>&gt=3B &gt=3B export P=
>>ATH=3D/usr/local/cm3/bin:$PATH<br>&gt=3B &gt=3B full CVS checkout <br>&g=
>>t=3B &gt=3B cd scripts/python =3B ./boot2.sh <br>&gt=3B &gt=3B <br>&gt=
>>=3B &gt=3B You will need to have made small changes to pylib.py for this =
>>to work.<br>&gt=3B &gt=3B (And your config file should say to use the C b=
>>ackend=2C like Darwin and<br>&gt=3B &gt=3B ARM_LINUX do.)<br>&gt=3B <br>&gt=
>>=3B <br>&gt=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t=
>>hat takes a long<br>&gt=3B time. Shouldn't FreeBSD use the C backend too?<=
>>br>&gt=3B <br>&gt=3B John<br></div> </div></body>
>></html>=
>>
>>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- 		 	   		  

From mika at async.caltech.edu  Mon Jan 20 00:39:38 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Sun, 19 Jan 2014 15:39:38 -0800
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>,
	<20140119201308.0A4B41A208F@async.async.caltech.edu>
	<COL130-W1536F70AC28586F319D481E6BA0@phx.gbl>
Message-ID: <20140119233938.9C5A31A208F@async.async.caltech.edu>

Jay K writes:
>The only problem I remember is an alignment matter using msvcrt.dll on AMD6=
>4_NT.=0A=
>I'll look into it=2C and the old emails.=0A=
>I've used the C backend on many platforms now.=0A=

This stuff doesn't happen with the native backend:

> 2. I can't get anything with Trestle working. I'm running in a VNC
> session on a remote machine. Not sure this has anything to do with the
> compiler or even the ARM. Has anyone seen it before (on any system)?
>
> (gdb) cont
> Continuing.
> [New Thread 0x42276430 (LWP 24451)]
> [Thread 0x42276430 (LWP 24451) exited]
> [New Thread 0x424c6430 (LWP 24454)]
>
>
> ***
> *** runtime error:
> *** Exception "ZChildVBT.BadPercentage" not in RAISES list
> *** file "../src/lego/ZChildVBT.m3", line 61
> ***
>
>
> Program received signal SIGABRT, Aborted.
>
> backtrace:
>
> (gdb) where
> #0 ZChildVBT__Init (z_L_120=0x422bbf70 "H\357K", ch_L_121=0x422bdb84 "\260\326K", h_L_122=0, v_L_123=1.75, loc_L_124=4 '\004', type_L_125=1 '\001', shaper_
L_126=0x41a32b54, open_L_127=0 '\000') at ../src/lego/ZCh
> ildVBT.m3:61
> #1 0x40637518 in FormsVBT__pZChild (cl_L_1042=0x42280444, list_L_1043=0x424c54c4, s_L_1044=0x424c55dc) at ../src/FormsVBT.m3:2593
> #2 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a45994 "", state_L_303=0x424c55dc) at ../src/FormsVBT.m3:250
> #3 0x40648b3c in FormsVBT__AddChildren (cl_L_1229=0x42280444, v_L_1230=0x422b0634 " \365K", list_L_1231=0x41a48504, state_L_1232=0x424c55dc) at ../src/FormsVBT.m3:3671
> #4 0x40634138 in FormsVBT__pZSplit (cl_L_999=0x42280444, list_L_1000=0x424c56bc, s_L_1001=0x424c57c4) at ../src/FormsVBT.m3:2454
> #5 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a40ec0 "", state_L_303=0x424c57c4) at ../src/FormsVBT.m3:250
> #6 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c57c4) at ../src/FormsVBT.m3:3642
> #7 0x406129c0 in FormsVBT__pShape (cl_L_496=0x42280444, list_L_497=0x424c58a4, s_L_498=0x424c59a4) at ../src/FormsVBT.m3:948
> #8 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a402a4 "", state_L_303=0x424c59a4) at ../src/FormsVBT.m3:250
> #9 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c59a4) at ../src/FormsVBT.m3:3642
> #10 0x4062ca10 in FormsVBT__pScale (cl_L_905=0x42280444, list_L_906=0x424c5a84, s_L_907=0x42280458) at ../src/FormsVBT.m3:2165
> #11 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a4006c "", state_L_303=0x42280458) at ../src/FormsVBT.m3:250
> #12 0x40606dec in FormsVBT__Apply (cl_L_293=0x42280444) at ../src/FormsVBT.m3:84
> #13 0x40d27bd4 in ThreadPThread__RunThread (me_L_231=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:449
> #14 0x40d27590 in ThreadPThread__ThreadBase (param_L_229=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:422
> #15 0x41850bfc in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
> #16 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> #17 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> (gdb) cont
> Continuing.
>



From jay.krell at cornell.edu  Mon Jan 20 01:06:27 2014
From: jay.krell at cornell.edu (Jay)
Date: Sun, 19 Jan 2014 16:06:27 -0800
Subject: [M3devel] Win32 SuspendThread?
In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL>
	<9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>
	<20140119200548.C18011A208F@async.async.caltech.edu>
Message-ID: <FC1A38F9-1A18-4DD1-99BB-9FB53669C761@gmail.com>

Cooperative suspend will also fix this signal allocation vs. valgrind problem. Can also likely pick a different one...

 - Jay

On Jan 19, 2014, at 12:05 PM, <mika at async.caltech.edu> wrote:

> "less worried" = "I think it works"?
> 
> I think the only threading that works in CM3 is user threading...
> 
> I tried to run CM3-compiled binaries through "valgrind" but got an
> error that CM3 is using the same signal that valgrind does internally
> (on AMD64_LINUX at least).  I think this is easy to change in m3core
> (yes?) but haven't gotten around to it...
> 
> valgrind has a threading checker called "helgrind"...
> 
>     Mika
> 
> Jay writes:
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/plain;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> For native I'm less worried now. For wow64 I'm still worried & hope to follo=
>> w up further.
>> 
>> - Jay
>> 
>> On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <rcolebur at SCIRES.COM> wrote:=
>> 
>> 
>>> Jay:
>>> =20
>>> I=E2=80=99m very concerned about the threading not working properly on bot=
>> h 32-bit and 64-bit Windows.
>>> =20
>>> The Thread Test program crashes for me on both platforms.
>>> =20
>>> I haven=E2=80=99t tried your new test program yet.
>>> =20
>>> --Randy
>>> =20
>>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>>> Sent: Saturday, January 18, 2014 1:23 AM
>>> To: m3devel
>>> Subject: EXT:[M3devel] Win32 SuspendThread?
>>> =20
>>> This program also doesn't behave as expected, native, nothing to do with w=
>> ow64.=20
>>> Anyone else please confirm:=20
>>>   1) my expectations -- it should never print anything
>>>   2) their importance  -- garbage collector depends on it =20
>>>   3) their not being met -- stuff gets printed=20
>>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20
>>> I am following up further.=20
>>> Maybe we should get cooperative suspend really going?=20
>>> Thank you.
>>> - Jay=20
>>> =20
>>> #include <stdio.h>
>>> #include <windows.h>
>>> volatile long value;
>>> unsigned long __stdcall Thread(PVOID parameter)
>>> {
>>>  while (1)
>>>   InterlockedIncrement(&value);
>>>  return 0;
>>> }
>>> int __cdecl main()
>>> {
>>>  HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>>  UINT i =3D 0;
>>>  while (1)
>>>  {
>>>    i +=3D 1;
>>>    if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>>    {
>>>      printf("suspend failed %X\n", GetLastError());
>>>      Sleep(1);
>>>      continue;
>>>    }
>>>    volatile long a =3D value;=20
>>>    volatile long b =3D value;
>>>    if (a !=3D b)
>>>    {
>>>     printf("%d %d %d %d\n", i, a, b, b - a);
>>>    }
>>>    ResumeThread(thread);
>>>  }
>>> }
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A
>> Content-Type: text/html;
>>    charset=utf-8
>> Content-Transfer-Encoding: quoted-printable
>> 
>> <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
>> utf-8"></head><body dir=3D"auto"><div>For native I'm less worried now. For w=
>> ow64 I'm still worried &amp; hope to follow up further.<br><br>&nbsp;- Jay</=
>> div><div><br>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" &lt;<a href=3D"=
>> mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM</a>&gt; wrote:<br><br></div>=
>> <blockquote type=3D"cite"><div>
>> 
>> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii">=
>> 
>> <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)">
>> <style><!--
>> /* Font Definitions */
>> @font-face
>>    {font-family:Calibri;
>>    panose-1:2 15 5 2 2 2 4 3 2 4;}
>> @font-face
>>    {font-family:Tahoma;
>>    panose-1:2 11 6 4 3 5 4 4 2 4;}
>> /* Style Definitions */
>> p.MsoNormal, li.MsoNormal, div.MsoNormal
>>    {margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> a:link, span.MsoHyperlink
>>    {mso-style-priority:99;
>>    color:blue;
>>    text-decoration:underline;}
>> a:visited, span.MsoHyperlinkFollowed
>>    {mso-style-priority:99;
>>    color:purple;
>>    text-decoration:underline;}
>> p
>>    {mso-style-priority:99;
>>    mso-margin-top-alt:auto;
>>    margin-right:0in;
>>    mso-margin-bottom-alt:auto;
>>    margin-left:0in;
>>    font-size:12.0pt;
>>    font-family:"Times New Roman","serif";}
>> p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
>>    {mso-style-priority:99;
>>    mso-style-link:"Balloon Text Char";
>>    margin:0in;
>>    margin-bottom:.0001pt;
>>    font-size:8.0pt;
>>    font-family:"Tahoma","sans-serif";}
>> span.EmailStyle18
>>    {mso-style-type:personal-reply;
>>    font-family:"Calibri","sans-serif";
>>    color:#1F497D;}
>> span.BalloonTextChar
>>    {mso-style-name:"Balloon Text Char";
>>    mso-style-priority:99;
>>    mso-style-link:"Balloon Text";
>>    font-family:"Tahoma","sans-serif";}
>> .MsoChpDefault
>>    {mso-style-type:export-only;
>>    font-size:10.0pt;}
>> @page WordSection1
>>    {size:8.5in 11.0in;
>>    margin:1.0in 1.0in 1.0in 1.0in;}
>> div.WordSection1
>>    {page:WordSection1;}
>> --></style><!--[if gte mso 9]><xml>
>> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
>> </xml><![endif]--><!--[if gte mso 9]><xml>
>> <o:shapelayout v:ext=3D"edit">
>> <o:idmap v:ext=3D"edit" data=3D"1" />
>> </o:shapelayout></xml><![endif]-->
>> 
>> 
>> <div class=3D"WordSection1">
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">Jay:<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I=E2=80=99m very concerned a=
>> bout the threading not working properly on both 32-bit and 64-bit Windows.<o=
>> :p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">The Thread Test program cra=
>> shes for me on both platforms.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">I haven=E2=80=99t tried you=
>> r new test program yet.<o:p></o:p></span></p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D">--Randy<o:p></o:p></span></=
>> p>
>> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:&quot;Cal=
>> ibri&quot;,&quot;sans-serif&quot;;color:#1F497D"><o:p>&nbsp;</o:p></span></p=
>> <div>
>> <div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0=
>> in 0in">
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-siz=
>> e:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">From:</span>=
>> </b><span style=3D"font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;san=
>> s-serif&quot;"> <a href=3D"mailto:jayk123 at hotmail.com">jayk123 at hotmail.com</=
>> a> [<a href=3D"mailto:jayk123 at hotmail.com">mailto:jayk123 at hotmail.com</a>]
>> <b>On Behalf Of </b>Jay K<br>
>> <b>Sent:</b> Saturday, January 18, 2014 1:23 AM<br>
>> <b>To:</b> m3devel<br>
>> <b>Subject:</b> EXT:[M3devel] Win32 SuspendThread?<o:p></o:p></span></p>
>> </div>
>> </div>
>> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p>&nbsp;</o:p></p>
>> <div>
>> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;marg=
>> in-bottom:12.0pt;margin-left:.5in">
>> <span style=3D"font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;">&nbsp=
>> ;This program also doesn't behave as expected, native, nothing to do with wo=
>> w64.
>> <br>
>> &nbsp;Anyone else please confirm: <br>
>> &nbsp;&nbsp; 1) my expectations -- it should never print anything<br>
>> &nbsp;&nbsp; 2) their importance&nbsp; -- garbage collector depends on it&nb=
>> sp; <br>
>> &nbsp; &nbsp;3) their not being met -- stuff gets printed <br>
>> &nbsp;This is in the CVS repository, scratch/wow64stack/sync2.cpp <br>
>> &nbsp;I am following up further. <br>
>> &nbsp;Maybe we should get cooperative suspend really going? <br>
>> Thank you.<br>
>> &nbsp;- Jay <br>
>> &nbsp;<br>
>> #include &lt;stdio.h&gt;<br>
>> #include &lt;windows.h&gt;<br>
>> volatile long value;<br>
>> unsigned long __stdcall Thread(PVOID parameter)<br>
>> {<br>
>> &nbsp; while (1)<br>
>> &nbsp;&nbsp; InterlockedIncrement(&amp;value);<br>
>> &nbsp; return 0;<br>
>> }<br>
>> int __cdecl main()<br>
>> {<br>
>> &nbsp; HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);<br>
>> &nbsp; UINT i =3D 0;<br>
>> &nbsp; while (1)<br>
>> &nbsp; {<br>
>> &nbsp;&nbsp;&nbsp; i +=3D 1;<br>
>> &nbsp;&nbsp;&nbsp; if (SuspendThread(thread) =3D=3D (DWORD)-1)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("suspend failed %X\n", GetLastError())=
>> ;<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sleep(1);<br>
>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; continue;<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; volatile long a =3D value; <br>
>> &nbsp;&nbsp;&nbsp; volatile long b =3D value;<br>
>> &nbsp;&nbsp;&nbsp; if (a !=3D b)<br>
>> &nbsp;&nbsp;&nbsp; {<br>
>> &nbsp;&nbsp;&nbsp;&nbsp; printf("%d %d %d %d\n", i, a, b, b - a);<br>
>> &nbsp;&nbsp;&nbsp; }<br>
>> &nbsp;&nbsp;&nbsp; ResumeThread(thread);<br>
>> &nbsp; }<br>
>> }<o:p></o:p></span></p>
>> </div>
>> </div>
>> 
>> 
>> </div></blockquote></body></html>=
>> 
>> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A--


From rodney_bates at lcwb.coop  Mon Jan 20 17:00:13 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 10:00:13 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD480D.3000101@lcwb.coop>

What does this do?

PROCEDURE Z(VAR x : REFANY) =
   VAR b: BDD.T;
   BEGIN
     b := x;
     x := BDD.Format(b)
   END Z;

The common property of the failing cases seems to be the necessity
of a runtime narrow check when passing x to BDD.Format.

So far, I haven't been able to reproduce the failure on AMD64_LINUX,
with even more irrelevant (one would think) stuff pared out.  I moved
T, Root, and Format into main, removed all but one fields of T and Root,
replaced BDD.New by NEW(Root) and removed Symtab and most of the body
from Format.

Looking at your code, I agree it should work.


On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From rodney_bates at lcwb.coop  Mon Jan 20 18:02:40 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 20 Jan 2014 11:02:40 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52DD56B0.1050007@lcwb.coop>

Lots more experiments, still no reproduction of the problem on AMD64_LINUX.
The 5 cases in the program below all work.  But earlier versions of this
test program, with bugs, (failure to repeat x:=y before every Z* call)
suggested a hypothesis.

For x := BDD.Format(x)

Perhaps the compiler does something to the LHS x (zeros it out?) before
narrowing and passing the x that is an actual parameter to Format.

If so, it is an implementation bug, as the language says (2.3.1):

   (for v := e)

   "The order of evaluation of v and e is undefined, but e will be evaluated
    before v is updated."

_____________________________________________________________________________________

MODULE Main;

TYPE T = OBJECT l : T; END;

TYPE Root = T OBJECT id : CARDINAL; END;

PROCEDURE Format(x : T) : TEXT =
   BEGIN
     RETURN "Format"
   END Format;

PROCEDURE P() : REFANY =
   BEGIN
     VAR x, y : REFANY;
     BEGIN
       y := NEW ( Root );
       x := y;
       Z1(x);
       x := y;
       Z2(x);
       x := y;
       Z3(x);
       x := y;
       Z4(x);
       x := y;
       Z5(x);
       RETURN x
     END
   END P;

PROCEDURE Z1(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T (b) => x := Format(b)
     END;
   END Z1;

PROCEDURE Z2(VAR x : REFANY) =
   VAR b : T;
   BEGIN
     b := x;
     x := Format(b)
   END Z2;

PROCEDURE Z3(VAR x : REFANY) =
   BEGIN
     x := Format(x)
   END Z3;

PROCEDURE Z4(VAR x : REFANY) =
   BEGIN
     TYPECASE x OF
       T => x := Format (x)
     END;
   END Z4;

PROCEDURE Z5(VAR x : REFANY) =
   BEGIN
     IF ISTYPE (x, T) THEN
       x := Format (x)
     END;
   END Z5;

BEGIN
   EVAL P()
END Main.

______________________________________________________________________________________

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Tue Jan 21 05:18:26 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Mon, 20 Jan 2014 20:18:26 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DD480D.3000101@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
Message-ID: <20140121041826.4D95D1A2080@async.async.caltech.edu>


Were you able to get it to fail on any of your computers?

Your Z fails for me---CM3 and PM3 both.

(579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany 


***
*** runtime error:
***    An explicit or implicit NARROW() operation failed.
***    file "../src/Main.m3", line 30
***

Abort
(580)truffles:~/bsd/refany/src>cat Mai
Main.m3~  Main.m3   
(580)truffles:~/bsd/refany/src>cat -n Main.m3
     1  MODULE Main;
     2  IMPORT BDD, IO;
     3
     4  PROCEDURE P() : REFANY =
     5    BEGIN
     6      VAR x : REFANY;
     7      BEGIN
     8        x := BDD.New("a");
     9        Z(x);
    10        RETURN x
    11      END
    12    END P;
    13
    14  PROCEDURE Q() =
    15    BEGIN
    16      IO.Put(P() & "\n")
    17    END Q;
    18
    19  (*
    20  PROCEDURE Z(VAR x : REFANY) =
    21    BEGIN
    22      TYPECASE x OF
    23        BDD.T(b) => x := BDD.Format(b)
    24      END;
    25    END Z;
    26  *)
    27  PROCEDURE Z(VAR x : REFANY) =
    28     VAR b: BDD.T;
    29     BEGIN
    30       b := x;
    31       x := BDD.Format(b)
    32     END Z;
    33
    34
    35
    36  BEGIN
    37    Q()
    38  END Main.
(581)truffles:~/bsd/refany/src>
"Rodney M. Bates" writes:
>What does this do?
>
>PROCEDURE Z(VAR x : REFANY) =
>   VAR b: BDD.T;
>   BEGIN
>     b := x;
>     x := BDD.Format(b)
>   END Z;
>
>The common property of the failing cases seems to be the necessity
>of a runtime narrow check when passing x to BDD.Format.
>
>So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>with even more irrelevant (one would think) stuff pared out.  I moved
>T, Root, and Format into main, removed all but one fields of T and Root,
>replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>from Format.
>
>Looking at your code, I agree it should work.
>
>
>On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>> As in the ISTYPE/TYPECASE example, the following works:
>>
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(b)
>>      END;
>>    END Z;
>>
>> (244)rover:~/refany/src>../FreeBSD4/refany
>> a
>>
>>
>> This does NOT work:
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    BEGIN
>>      TYPECASE x OF
>>        BDD.T(b) => x := BDD.Format(x)
>>      END;
>>    END Z;
>>
>>
>> (242)rover:~/refany/src>../FreeBSD4/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    NARROW failed
>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>> ***
>>
>>    use option @M3stackdump to get a stack trace
>> Abort
>>
>>
>> Same result PM3 / CM3.
>>
>>       Mika
>>


From rodney_bates at lcwb.coop  Tue Jan 21 17:34:57 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 10:34:57 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEA1B1.2070808@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?

No, the only times I got narrow failures were due to errors
in my trial cases.

I have only tried AMD64_LINUX, but should be able to get this onto a
LINUXLIBC6 system.

>
> Your Z fails for me---CM3 and PM3 both.
>
> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>
>
> ***
> *** runtime error:
> ***    An explicit or implicit NARROW() operation failed.
> ***    file "../src/Main.m3", line 30
> ***

Well that torpedos both my theories.

But it is hard to imagine that bad code generated for an implicit narrow
on an assignment could ever have gone unnoticed so long.  I thought it
might be more believable if it only happened when narrowing an actual
parameter before passing it.

>
> Abort
> (580)truffles:~/bsd/refany/src>cat Mai
> Main.m3~  Main.m3
> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>       1  MODULE Main;
>       2  IMPORT BDD, IO;
>       3
>       4  PROCEDURE P() : REFANY =
>       5    BEGIN
>       6      VAR x : REFANY;
>       7      BEGIN
>       8        x := BDD.New("a");
>       9        Z(x);
>      10        RETURN x
>      11      END
>      12    END P;
>      13
>      14  PROCEDURE Q() =
>      15    BEGIN
>      16      IO.Put(P() & "\n")
>      17    END Q;
>      18
>      19  (*
>      20  PROCEDURE Z(VAR x : REFANY) =
>      21    BEGIN
>      22      TYPECASE x OF
>      23        BDD.T(b) => x := BDD.Format(b)
>      24      END;
>      25    END Z;
>      26  *)
>      27  PROCEDURE Z(VAR x : REFANY) =
>      28     VAR b: BDD.T;
>      29     BEGIN
>      30       b := x;
>      31       x := BDD.Format(b)
>      32     END Z;
>      33
>      34
>      35
>      36  BEGIN
>      37    Q()
>      38  END Main.
> (581)truffles:~/bsd/refany/src>
> "Rodney M. Bates" writes:
>> What does this do?
>>
>> PROCEDURE Z(VAR x : REFANY) =
>>    VAR b: BDD.T;
>>    BEGIN
>>      b := x;
>>      x := BDD.Format(b)
>>    END Z;
>>
>> The common property of the failing cases seems to be the necessity
>> of a runtime narrow check when passing x to BDD.Format.
>>
>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>> with even more irrelevant (one would think) stuff pared out.  I moved
>> T, Root, and Format into main, removed all but one fields of T and Root,
>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>from Format.
>>
>> Looking at your code, I agree it should work.
>>
>>
>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>> As in the ISTYPE/TYPECASE example, the following works:
>>>
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(b)
>>>       END;
>>>     END Z;
>>>
>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>> a
>>>
>>>
>>> This does NOT work:
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>     BEGIN
>>>       TYPECASE x OF
>>>         BDD.T(b) => x := BDD.Format(x)
>>>       END;
>>>     END Z;
>>>
>>>
>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    NARROW failed
>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>> ***
>>>
>>>     use option @M3stackdump to get a stack trace
>>> Abort
>>>
>>>
>>> Same result PM3 / CM3.
>>>
>>>        Mika
>>>
>



From mika at async.caltech.edu  Tue Jan 21 18:11:20 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 09:11:20 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEA1B1.2070808@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
Message-ID: <20140121171120.F21FF1A2080@async.async.caltech.edu>

Did you try using my bdd.tgz ?

I am getting this error on: 

ancient PM3 on FreeBSD4

head CM3 on AMD64_LINUX

few months old CM3 on ARM_LINUX (Raspberry Pi)

I doubt you will get it on LINUXLIBC6 if you don't get it on the others.

I already tried rearranging the files (the implementation and interfaces
are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
right now).  With no effect.  I don't know what it is about my code that's
tripping up the compiler.

It's also a "decent" BDD package so someone (me I guess, but anyone
else who's interested is welcome to) might want to add it to CM3... :-)
Kind of a pain if you can't print out the debugging info though!
(Actually the workarounds are OK.)

    Mika

"Rodney M. Bates" writes:
>
>
>On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>> Were you able to get it to fail on any of your computers?
>
>No, the only times I got narrow failures were due to errors
>in my trial cases.
>
>I have only tried AMD64_LINUX, but should be able to get this onto a
>LINUXLIBC6 system.
>
>>
>> Your Z fails for me---CM3 and PM3 both.
>>
>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>
>>
>> ***
>> *** runtime error:
>> ***    An explicit or implicit NARROW() operation failed.
>> ***    file "../src/Main.m3", line 30
>> ***
>
>Well that torpedos both my theories.
>
>But it is hard to imagine that bad code generated for an implicit narrow
>on an assignment could ever have gone unnoticed so long.  I thought it
>might be more believable if it only happened when narrowing an actual
>parameter before passing it.
>
>>
>> Abort
>> (580)truffles:~/bsd/refany/src>cat Mai
>> Main.m3~  Main.m3
>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>       1  MODULE Main;
>>       2  IMPORT BDD, IO;
>>       3
>>       4  PROCEDURE P() : REFANY =
>>       5    BEGIN
>>       6      VAR x : REFANY;
>>       7      BEGIN
>>       8        x := BDD.New("a");
>>       9        Z(x);
>>      10        RETURN x
>>      11      END
>>      12    END P;
>>      13
>>      14  PROCEDURE Q() =
>>      15    BEGIN
>>      16      IO.Put(P() & "\n")
>>      17    END Q;
>>      18
>>      19  (*
>>      20  PROCEDURE Z(VAR x : REFANY) =
>>      21    BEGIN
>>      22      TYPECASE x OF
>>      23        BDD.T(b) => x := BDD.Format(b)
>>      24      END;
>>      25    END Z;
>>      26  *)
>>      27  PROCEDURE Z(VAR x : REFANY) =
>>      28     VAR b: BDD.T;
>>      29     BEGIN
>>      30       b := x;
>>      31       x := BDD.Format(b)
>>      32     END Z;
>>      33
>>      34
>>      35
>>      36  BEGIN
>>      37    Q()
>>      38  END Main.
>> (581)truffles:~/bsd/refany/src>
>> "Rodney M. Bates" writes:
>>> What does this do?
>>>
>>> PROCEDURE Z(VAR x : REFANY) =
>>>    VAR b: BDD.T;
>>>    BEGIN
>>>      b := x;
>>>      x := BDD.Format(b)
>>>    END Z;
>>>
>>> The common property of the failing cases seems to be the necessity
>>> of a runtime narrow check when passing x to BDD.Format.
>>>
>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>from Format.
>>>
>>> Looking at your code, I agree it should work.
>>>
>>>
>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(b)
>>>>       END;
>>>>     END Z;
>>>>
>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>> a
>>>>
>>>>
>>>> This does NOT work:
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     BEGIN
>>>>       TYPECASE x OF
>>>>         BDD.T(b) => x := BDD.Format(x)
>>>>       END;
>>>>     END Z;
>>>>
>>>>
>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    NARROW failed
>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>> ***
>>>>
>>>>     use option @M3stackdump to get a stack trace
>>>> Abort
>>>>
>>>>
>>>> Same result PM3 / CM3.
>>>>
>>>>        Mika
>>>>
>>


From rodney_bates at lcwb.coop  Tue Jan 21 18:23:41 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 11:23:41 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
Message-ID: <52DEAD1D.8010602@lcwb.coop>



On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
> Were you able to get it to fail on any of your computers?
>

My test case with 5 versions of Z, and your complete one that you posted
(bdd.tgz) both silently complete with no error messages on LINUXLIBC6,
using the head compiler and libraries.

bdd.tgz also works fine on AMD64_LINUX.

On yours, I built and shipped in bdd/src, then built in bdd/test/src
and ran LINUXLIBC6/bddtest, which I presume was the right procedure.

How about compiling a couple of failing Z versions on a machine where
they fail, with -keep, then posting the .mc, .ms, and .mo files and corresponding
source (only for the Zs is probably enough).


>>>        Mika
>>>
>



From rodney_bates at lcwb.coop  Tue Jan 21 19:02:09 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Tue, 21 Jan 2014 12:02:09 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140121171120.F21FF1A2080@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
Message-ID: <52DEB621.7080706@lcwb.coop>



On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
> Did you try using my bdd.tgz ?
>
> I am getting this error on:
>
> ancient PM3 on FreeBSD4
>
> head CM3 on AMD64_LINUX

Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
slightly different in size when I recompiled, but I think there is an explanation
for that.

I can't build bddtest with the release compiler, because it needs caltech_parser,
and that won't build in the release.  This combination of [non]failures seems
pretty strange.  I am also wondering about a problem with type checks in the
runtime.


>
> few months old CM3 on ARM_LINUX (Raspberry Pi)
>
> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>
> I already tried rearranging the files (the implementation and interfaces
> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
> right now).  With no effect.  I don't know what it is about my code that's
> tripping up the compiler.
>
> It's also a "decent" BDD package so someone (me I guess, but anyone
> else who's interested is welcome to) might want to add it to CM3... :-)
> Kind of a pain if you can't print out the debugging info though!
> (Actually the workarounds are OK.)
>
>      Mika
>
> "Rodney M. Bates" writes:
>>
>>
>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>> Were you able to get it to fail on any of your computers?
>>
>> No, the only times I got narrow failures were due to errors
>> in my trial cases.
>>
>> I have only tried AMD64_LINUX, but should be able to get this onto a
>> LINUXLIBC6 system.
>>
>>>
>>> Your Z fails for me---CM3 and PM3 both.
>>>
>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>
>>>
>>> ***
>>> *** runtime error:
>>> ***    An explicit or implicit NARROW() operation failed.
>>> ***    file "../src/Main.m3", line 30
>>> ***
>>
>> Well that torpedos both my theories.
>>
>> But it is hard to imagine that bad code generated for an implicit narrow
>> on an assignment could ever have gone unnoticed so long.  I thought it
>> might be more believable if it only happened when narrowing an actual
>> parameter before passing it.
>>
>>>
>>> Abort
>>> (580)truffles:~/bsd/refany/src>cat Mai
>>> Main.m3~  Main.m3
>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>        1  MODULE Main;
>>>        2  IMPORT BDD, IO;
>>>        3
>>>        4  PROCEDURE P() : REFANY =
>>>        5    BEGIN
>>>        6      VAR x : REFANY;
>>>        7      BEGIN
>>>        8        x := BDD.New("a");
>>>        9        Z(x);
>>>       10        RETURN x
>>>       11      END
>>>       12    END P;
>>>       13
>>>       14  PROCEDURE Q() =
>>>       15    BEGIN
>>>       16      IO.Put(P() & "\n")
>>>       17    END Q;
>>>       18
>>>       19  (*
>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>       21    BEGIN
>>>       22      TYPECASE x OF
>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>       24      END;
>>>       25    END Z;
>>>       26  *)
>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>       28     VAR b: BDD.T;
>>>       29     BEGIN
>>>       30       b := x;
>>>       31       x := BDD.Format(b)
>>>       32     END Z;
>>>       33
>>>       34
>>>       35
>>>       36  BEGIN
>>>       37    Q()
>>>       38  END Main.
>>> (581)truffles:~/bsd/refany/src>
>>> "Rodney M. Bates" writes:
>>>> What does this do?
>>>>
>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>     VAR b: BDD.T;
>>>>     BEGIN
>>>>       b := x;
>>>>       x := BDD.Format(b)
>>>>     END Z;
>>>>
>>>> The common property of the failing cases seems to be the necessity
>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>
>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>> >from Format.
>>>>
>>>> Looking at your code, I agree it should work.
>>>>
>>>>
>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>> a
>>>>>
>>>>>
>>>>> This does NOT work:
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>      BEGIN
>>>>>        TYPECASE x OF
>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>        END;
>>>>>      END Z;
>>>>>
>>>>>
>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>
>>>>>
>>>>> ***
>>>>> *** runtime error:
>>>>> ***    NARROW failed
>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>> ***
>>>>>
>>>>>      use option @M3stackdump to get a stack trace
>>>>> Abort
>>>>>
>>>>>
>>>>> Same result PM3 / CM3.
>>>>>
>>>>>         Mika
>>>>>
>>>
>



From mika at async.caltech.edu  Tue Jan 21 19:08:06 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 10:08:06 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52DEB621.7080706@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
Message-ID: <20140121180806.D75A61A208F@async.async.caltech.edu>

The only thing bddtest uses from caltech_parser is Debug.Out.  You can just take out those calls or replace them with IO.Put.

But again bddtest is not the program that fails... it's the "refany" program (now in a sibling directory to bddtest)

   Mika

"Rodney M. Bates" writes:
>
>
>On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>> Did you try using my bdd.tgz ?
>>
>> I am getting this error on:
>>
>> ancient PM3 on FreeBSD4
>>
>> head CM3 on AMD64_LINUX
>
>Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with
>a freshly updated head.  Hacking around, I tried to use the binaries in bdd.tgz,
>but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files are
>slightly different in size when I recompiled, but I think there is an explanation
>for that.
>
>I can't build bddtest with the release compiler, because it needs caltech_parser,
>and that won't build in the release.  This combination of [non]failures seems
>pretty strange.  I am also wondering about a problem with type checks in the
>runtime.
>
>
>>
>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>
>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.
>>
>> I already tried rearranging the files (the implementation and interfaces
>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>> right now).  With no effect.  I don't know what it is about my code that's
>> tripping up the compiler.
>>
>> It's also a "decent" BDD package so someone (me I guess, but anyone
>> else who's interested is welcome to) might want to add it to CM3... :-)
>> Kind of a pain if you can't print out the debugging info though!
>> (Actually the workarounds are OK.)
>>
>>      Mika
>>
>> "Rodney M. Bates" writes:
>>>
>>>
>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>> Were you able to get it to fail on any of your computers?
>>>
>>> No, the only times I got narrow failures were due to errors
>>> in my trial cases.
>>>
>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>> LINUXLIBC6 system.
>>>
>>>>
>>>> Your Z fails for me---CM3 and PM3 both.
>>>>
>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> ***    An explicit or implicit NARROW() operation failed.
>>>> ***    file "../src/Main.m3", line 30
>>>> ***
>>>
>>> Well that torpedos both my theories.
>>>
>>> But it is hard to imagine that bad code generated for an implicit narrow
>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>> might be more believable if it only happened when narrowing an actual
>>> parameter before passing it.
>>>
>>>>
>>>> Abort
>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>> Main.m3~  Main.m3
>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>        1  MODULE Main;
>>>>        2  IMPORT BDD, IO;
>>>>        3
>>>>        4  PROCEDURE P() : REFANY =
>>>>        5    BEGIN
>>>>        6      VAR x : REFANY;
>>>>        7      BEGIN
>>>>        8        x := BDD.New("a");
>>>>        9        Z(x);
>>>>       10        RETURN x
>>>>       11      END
>>>>       12    END P;
>>>>       13
>>>>       14  PROCEDURE Q() =
>>>>       15    BEGIN
>>>>       16      IO.Put(P() & "\n")
>>>>       17    END Q;
>>>>       18
>>>>       19  (*
>>>>       20  PROCEDURE Z(VAR x : REFANY) =
>>>>       21    BEGIN
>>>>       22      TYPECASE x OF
>>>>       23        BDD.T(b) => x := BDD.Format(b)
>>>>       24      END;
>>>>       25    END Z;
>>>>       26  *)
>>>>       27  PROCEDURE Z(VAR x : REFANY) =
>>>>       28     VAR b: BDD.T;
>>>>       29     BEGIN
>>>>       30       b := x;
>>>>       31       x := BDD.Format(b)
>>>>       32     END Z;
>>>>       33
>>>>       34
>>>>       35
>>>>       36  BEGIN
>>>>       37    Q()
>>>>       38  END Main.
>>>> (581)truffles:~/bsd/refany/src>
>>>> "Rodney M. Bates" writes:
>>>>> What does this do?
>>>>>
>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>     VAR b: BDD.T;
>>>>>     BEGIN
>>>>>       b := x;
>>>>>       x := BDD.Format(b)
>>>>>     END Z;
>>>>>
>>>>> The common property of the failing cases seems to be the necessity
>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>
>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>> with even more irrelevant (one would think) stuff pared out.  I moved
>>>>> T, Root, and Format into main, removed all but one fields of T and Root,
>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body
>>>> >from Format.
>>>>>
>>>>> Looking at your code, I agree it should work.
>>>>>
>>>>>
>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(b)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>> a
>>>>>>
>>>>>>
>>>>>> This does NOT work:
>>>>>>
>>>>>> PROCEDURE Z(VAR x : REFANY) =
>>>>>>      BEGIN
>>>>>>        TYPECASE x OF
>>>>>>          BDD.T(b) => x := BDD.Format(x)
>>>>>>        END;
>>>>>>      END Z;
>>>>>>
>>>>>>
>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>
>>>>>>
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    NARROW failed
>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>> ***
>>>>>>
>>>>>>      use option @M3stackdump to get a stack trace
>>>>>> Abort
>>>>>>
>>>>>>
>>>>>> Same result PM3 / CM3.
>>>>>>
>>>>>>         Mika
>>>>>>
>>>>
>>


From mika at async.caltech.edu  Tue Jan 21 20:23:28 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 21 Jan 2014 11:23:28 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52DD480D.3000101@lcwb.coop>
	<20140121041826.4D95D1A2080@async.async.caltech.edu>
	<52DEA1B1.2070808@lcwb.coop>
	<20140121171120.F21FF1A2080@async.async.caltech.edu>
	<52DEB621.7080706@lcwb.coop>
	<20140121180806.D75A61A208F@async.async.caltech.edu>
	<07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu>
Message-ID: <20140121192328.92DDB1A208F@async.async.caltech.edu>

How do you get them into text form?  They look binary...

I copied the whole directory bdd (and all subdirs) to 

http://rover.gcapltd.com/~mika/bdd/

Antony Hosking writes:
>Please post .io / .mo in text form.
>
>Sent from my iPad
>
>> On Jan 21, 2014, at 1:08 PM, mika at async.caltech.edu wrote:
>>=20
>> The only thing bddtest uses from caltech_parser is Debug.Out.  You can jus=
>t take out those calls or replace them with IO.Put.
>>=20
>> But again bddtest is not the program that fails... it's the "refany" progr=
>am (now in a sibling directory to bddtest)
>>=20
>>   Mika
>>=20
>> "Rodney M. Bates" writes:
>>>=20
>>>=20
>>>> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote:
>>>> Did you try using my bdd.tgz ?
>>>>=20
>>>> I am getting this error on:
>>>>=20
>>>> ancient PM3 on FreeBSD4
>>>>=20
>>>> head CM3 on AMD64_LINUX
>>>=20
>>> Hmm.  I can't get any errors compiling bdd and bddtest on my AMD64_LINUX s=
>ystem with
>>> a freshly updated head.  Hacking around, I tried to use the binaries in b=
>dd.tgz,
>>> but my .M3SHIP wants a libbdd.so.5, not there.  Some of the *.[im]o files=
> are
>>> slightly different in size when I recompiled, but I think there is an exp=
>lanation
>>> for that.
>>>=20
>>> I can't build bddtest with the release compiler, because it needs caltech=
>_parser,
>>> and that won't build in the release.  This combination of [non]failures s=
>eems
>>> pretty strange.  I am also wondering about a problem with type checks in t=
>he
>>> runtime.
>>>=20
>>>=20
>>>>=20
>>>> few months old CM3 on ARM_LINUX (Raspberry Pi)
>>>>=20
>>>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.=
>
>>>>=20
>>>> I already tried rearranging the files (the implementation and interfaces=
>
>>>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3
>>>> right now).  With no effect.  I don't know what it is about my code that=
>'s
>>>> tripping up the compiler.
>>>>=20
>>>> It's also a "decent" BDD package so someone (me I guess, but anyone
>>>> else who's interested is welcome to) might want to add it to CM3... :-)
>>>> Kind of a pain if you can't print out the debugging info though!
>>>> (Actually the workarounds are OK.)
>>>>=20
>>>>     Mika
>>>>=20
>>>> "Rodney M. Bates" writes:
>>>>>=20
>>>>>=20
>>>>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote:
>>>>>> Were you able to get it to fail on any of your computers?
>>>>>=20
>>>>> No, the only times I got narrow failures were due to errors
>>>>> in my trial cases.
>>>>>=20
>>>>> I have only tried AMD64_LINUX, but should be able to get this onto a
>>>>> LINUXLIBC6 system.
>>>>>=20
>>>>>>=20
>>>>>> Your Z fails for me---CM3 and PM3 both.
>>>>>>=20
>>>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany
>>>>>>=20
>>>>>>=20
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    An explicit or implicit NARROW() operation failed.
>>>>>> ***    file "../src/Main.m3", line 30
>>>>>> ***
>>>>>=20
>>>>> Well that torpedos both my theories.
>>>>>=20
>>>>> But it is hard to imagine that bad code generated for an implicit narro=
>w
>>>>> on an assignment could ever have gone unnoticed so long.  I thought it
>>>>> might be more believable if it only happened when narrowing an actual
>>>>> parameter before passing it.
>>>>>=20
>>>>>>=20
>>>>>> Abort
>>>>>> (580)truffles:~/bsd/refany/src>cat Mai
>>>>>> Main.m3~  Main.m3
>>>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3
>>>>>>       1  MODULE Main;
>>>>>>       2  IMPORT BDD, IO;
>>>>>>       3
>>>>>>       4  PROCEDURE P() : REFANY =3D
>>>>>>       5    BEGIN
>>>>>>       6      VAR x : REFANY;
>>>>>>       7      BEGIN
>>>>>>       8        x :=3D BDD.New("a");
>>>>>>       9        Z(x);
>>>>>>      10        RETURN x
>>>>>>      11      END
>>>>>>      12    END P;
>>>>>>      13
>>>>>>      14  PROCEDURE Q() =3D
>>>>>>      15    BEGIN
>>>>>>      16      IO.Put(P() & "\n")
>>>>>>      17    END Q;
>>>>>>      18
>>>>>>      19  (*
>>>>>>      20  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      21    BEGIN
>>>>>>      22      TYPECASE x OF
>>>>>>      23        BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>      24      END;
>>>>>>      25    END Z;
>>>>>>      26  *)
>>>>>>      27  PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>      28     VAR b: BDD.T;
>>>>>>      29     BEGIN
>>>>>>      30       b :=3D x;
>>>>>>      31       x :=3D BDD.Format(b)
>>>>>>      32     END Z;
>>>>>>      33
>>>>>>      34
>>>>>>      35
>>>>>>      36  BEGIN
>>>>>>      37    Q()
>>>>>>      38  END Main.
>>>>>> (581)truffles:~/bsd/refany/src>
>>>>>> "Rodney M. Bates" writes:
>>>>>>> What does this do?
>>>>>>>=20
>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>    VAR b: BDD.T;
>>>>>>>    BEGIN
>>>>>>>      b :=3D x;
>>>>>>>      x :=3D BDD.Format(b)
>>>>>>>    END Z;
>>>>>>>=20
>>>>>>> The common property of the failing cases seems to be the necessity
>>>>>>> of a runtime narrow check when passing x to BDD.Format.
>>>>>>>=20
>>>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX,
>>>>>>> with even more irrelevant (one would think) stuff pared out.  I moved=
>
>>>>>>> T, Root, and Format into main, removed all but one fields of T and Ro=
>ot,
>>>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body=
>
>>>>>>> from Format.
>>>>>>>=20
>>>>>>> Looking at your code, I agree it should work.
>>>>>>>=20
>>>>>>>=20
>>>>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
>>>>>>>> As in the ISTYPE/TYPECASE example, the following works:
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(b)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>> a
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> This does NOT work:
>>>>>>>>=20
>>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D
>>>>>>>>     BEGIN
>>>>>>>>       TYPECASE x OF
>>>>>>>>         BDD.T(b) =3D> x :=3D BDD.Format(x)
>>>>>>>>       END;
>>>>>>>>     END Z;
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> ***
>>>>>>>> *** runtime error:
>>>>>>>> ***    NARROW failed
>>>>>>>> ***    file "/big/home/mika/refany/src/Main.m3", line 22
>>>>>>>> ***
>>>>>>>>=20
>>>>>>>>     use option @M3stackdump to get a stack trace
>>>>>>>> Abort
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Same result PM3 / CM3.
>>>>>>>>=20
>>>>>>>>        Mika
>>>>=20


From rodney_bates at lcwb.coop  Wed Jan 22 20:46:55 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Wed, 22 Jan 2014 13:46:55 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
Message-ID: <52E0202F.3020307@lcwb.coop>

Some more info:

The front end is generating incorrect narrow code, only
where the type T is an unrevealed <: REFANY.  When T and Root
are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
and implicit narrow in an assignment.

When opaque, for explicit and implicit narrow, it simply compares
the typecode of x to a single typecode (no doubt for T) and demands
they be equal.  In this case, the actual allocated type of x is Root,
a proper subtype of T.

For ISTYPE, it does call CheckIsType even when the types are opaque.

The fact that it happens only when the types are opaque lends
credibility to the fact that this bug has gone unnoticed at least
as far back as PM3.

On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote:
> As in the ISTYPE/TYPECASE example, the following works:
>
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(b)
>      END;
>    END Z;
>
> (244)rover:~/refany/src>../FreeBSD4/refany
> a
>
>
> This does NOT work:
>
> PROCEDURE Z(VAR x : REFANY) =
>    BEGIN
>      TYPECASE x OF
>        BDD.T(b) => x := BDD.Format(x)
>      END;
>    END Z;
>
>
> (242)rover:~/refany/src>../FreeBSD4/refany
>
>
> ***
> *** runtime error:
> ***    NARROW failed
> ***    file "/big/home/mika/refany/src/Main.m3", line 22
> ***
>
>    use option @M3stackdump to get a stack trace
> Abort
>
>
> Same result PM3 / CM3.
>
>       Mika
>



From mika at async.caltech.edu  Wed Jan 22 21:47:49 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Wed, 22 Jan 2014 12:47:49 -0800
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E0202F.3020307@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
Message-ID: <20140122204749.C724E1A208F@async.async.caltech.edu>


Aha so declaring the type as <: ROOT will probably work as a workaround?

Thank you for looking into this.

"Rodney M. Bates" writes:
>Some more info:
>
>The front end is generating incorrect narrow code, only
>where the type T is an unrevealed <: REFANY.  When T and Root
>are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>and implicit narrow in an assignment.
>
>When opaque, for explicit and implicit narrow, it simply compares
>the typecode of x to a single typecode (no doubt for T) and demands
>they be equal.  In this case, the actual allocated type of x is Root,
>a proper subtype of T.
>
>For ISTYPE, it does call CheckIsType even when the types are opaque.
>
>The fact that it happens only when the types are opaque lends
>credibility to the fact that this bug has gone unnoticed at least
>as far back as PM3.


From jay.krell at cornell.edu  Thu Jan 23 08:51:58 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 07:51:58 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/046e93c5/attachment-0002.html>

From rcolebur at SCIRES.COM  Thu Jan 23 12:59:34 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 11:59:34 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
Message-ID: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:

  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/ab6f0152/attachment-0002.html>

From rodney_bates at lcwb.coop  Thu Jan 23 16:25:59 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Thu, 23 Jan 2014 09:25:59 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <20140122204749.C724E1A208F@async.async.caltech.edu>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
Message-ID: <52E13487.9090504@lcwb.coop>



On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
> Aha so declaring the type as <: ROOT will probably work as a workaround?
>

I'm not sure whether that would do it or not.  In my pared-down case, I had
moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
the failures go away.

Until it is fixed, I think the best workaround is TYPECASE with a binding:

TYPECASE x
OF BDD.T(b) =>
   Format(b), etc.

This is known to be working now.  You've probably thought about this, but
this way requires only a single runtime type test.  Either TYPECASE without
the binding of b, or ISTYPE, will require an additional recheck of the same
thing when the narrow, implicit or explicit, happens.  I doubt even a
pretty good optimizing compiler could remove this.  For ISTYPE, it would have
to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
relation between CheckIsType and ScanTypecase.

> Thank you for looking into this.
>
> "Rodney M. Bates" writes:
>> Some more info:
>>
>> The front end is generating incorrect narrow code, only
>> where the type T is an unrevealed <: REFANY.  When T and Root
>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>> and implicit narrow in an assignment.
>>
>> When opaque, for explicit and implicit narrow, it simply compares
>> the typecode of x to a single typecode (no doubt for T) and demands
>> they be equal.  In this case, the actual allocated type of x is Root,
>> a proper subtype of T.
>>
>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>
>> The fact that it happens only when the types are opaque lends
>> credibility to the fact that this bug has gone unnoticed at least
>> as far back as PM3.
>



From hosking at cs.purdue.edu  Thu Jan 23 17:51:02 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Thu, 23 Jan 2014 11:51:02 -0500
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
	reliably?
In-Reply-To: <C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>
Message-ID: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:

> Jay,
> 
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
> 
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
> 
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
> 
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
> 
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
> 
>>   There is a behavior/bug in wow64.  
>>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
>> 
>> 
>> 
>>   SuspendThread / GetThreadContext work like this:  
>> 
>> 
>>   32bit processes consist almost entirely of 32bit code.  
>>   There is a small amount of 64bit code.  
>> 
>> 
>>   If you suspend while running 32bit code, GetThreadContext works.  
>>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
>> 
>> 
>>   64bit code is run en route to syscalls.  
>>   For example you call:  
>>     1 kernel32!Sleep   
>>     2 it calls 32bit NtDelayExecution   
>>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>>     4 which calls native NtDelayExecuation   
>> 
>>  
>>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>>     You can step through it very easily in a debugger. Really. 
>>   Where GetThreadContext knows where to get it.  
>>   The problem is that saving context is not atomic.  
>>   You can suspend while saving context.  
>> 
>> 
>>   What to do? 
>> 
>> 
>>   scratch/wow64stack contains a program that detects the bug.
>>   I believe it is the basis of a workaround for the bug.   
>> 
>> 
>>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>>   not only functions that use exception handling, but also functions that call "extern" functions  
>>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>>   use that, if it isn't zero. 
>>   Normally it will be zero.  
>>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>>   That is why it isn't a set/set-zero pattern (like in the test program).   
>> 
>> 
>>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
>> 
>> 
>>   Rejected counter proposal: 
>>    Don't suspend/gc when running a syscall.  
>>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
>> 
>> 
>>  Possible augmentation: if running native, short circuit most of this 
>> 
>> 
>>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
>> 
>> 
>>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
>> 
>> 
>>   Performance impact: hypothetically large but probably not noticable. 
>> 
>> 
>>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>>   We could further augment pragmas to discern them. 
>> 
>> 
>>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
>> 
>> 
>> 
>>   Agreed? I'll make the compiler change? 
>> 
>> 
>> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
>> 
>> 
>> Eventually cooperative suspend will cause this to fall away as a problem..
>> 
>> 
>>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/8a7dd0a9/attachment-0002.html>

From jay.krell at cornell.edu  Thu Jan 23 21:03:09 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 20:03:09 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>,
	<COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>
Message-ID: <COL130-W24D43515B172BD96CF08F6E6A60@phx.gbl>

Syscalls are "doubly wrapped".
Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention.
Lesser known, there are also some syscalls in gdi32.dll and user32.dll.
None of them are public interfaces.
They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll.
For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change.
Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work.

There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall.

Bottom line is though, one isn't really supposed to know where the syscalls are.

 - Jay

From: jay.krell at cornell.edu
To: hosking at cs.purdue.edu; rcolebur at scires.com
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
Date: Thu, 23 Jan 2014 19:57:27 +0000




There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/7b51ac64/attachment-0002.html>

From jay.krell at cornell.edu  Thu Jan 23 20:57:27 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 19:57:27 +0000
Subject: [M3devel] EXT: proposal for getting wow64 context/stack
 reliably?
In-Reply-To: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
References: <COL130-W4907959865CC284A4C1E5CE6A60@phx.gbl>,
	<C286F56A-3CBC-4F04-8AD2-B5EB3C41B1EE@SCIRES.COM>,
	<4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>
Message-ID: <COL130-W76D1B70F8CB3C37130D1AEE6A60@phx.gbl>

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack	reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows).
However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
Are there any linker tricks for Windows one can use to wrap only system calls?
I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:Jay,
If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
--Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:

  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5f6fec82/attachment-0002.html>

From rcolebur at SCIRES.COM  Thu Jan 23 23:23:43 2014
From: rcolebur at SCIRES.COM (Coleburn, Randy)
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>

Jay:

When you say native 32/64-bit "should be ok", you really mean ok with respect to the SYSWOW64 problem you've identified.

Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?

--Randy

From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - Jay
________________________________
From: hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
CC: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>; jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.
We need to address the native thread problem on all platforms (both pthread and windows).

However, this particular problem is pernicious if we need to save M3 stack state at all external calls.

Are there any linker tricks for Windows one can use to wrap only system calls?

I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.

It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484






On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>> wrote:

Jay,

If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.

Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.

Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.

Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?

--Randy

Sent from my iPhone

On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>> wrote:
  There is a behavior/bug in wow64.
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:


  32bit processes consist almost entirely of 32bit code.
  There is a small amount of 64bit code.


  If you suspend while running 32bit code, GetThreadContext works.
  if you suspend while running 64bit code, GetThreadContext usually but not always works.


  64bit code is run en route to syscalls.
  For example you call:
    1 kernel32!Sleep
    2 it calls 32bit NtDelayExecution
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call)
    4 which calls native NtDelayExecuation


  In between 2 and 3, within 64bit code, the 32bit context is saved.
    You can step through it very easily in a debugger. Really.
  Where GetThreadContext knows where to get it.
  The problem is that saving context is not atomic.
  You can suspend while saving context.


  What to do?


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,
  not only functions that use exception handling, but also functions that call "extern" functions
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero.
  Normally it will be zero.
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.
  That is why it isn't a set/set-zero pattern (like in the test program).


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.


  Rejected counter proposal:
   Don't suspend/gc when running a syscall.
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.


 Possible augmentation: if running native, short circuit most of this


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,
   until/unless there is another 32bit platform runable on some similar 64bit platforms.


  Performance impact: hypothetically large but probably not noticable.


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them.


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...



  Agreed? I'll make the compiler change?


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/c534e985/attachment-0002.html>

From jay.krell at cornell.edu  Fri Jan 24 00:09:50 2014
From: jay.krell at cornell.edu (Jay K)
Date: Thu, 23 Jan 2014 23:09:50 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
Message-ID: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>

Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?









Jay:
 
When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 

 
Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
 
--Randy
 


From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com]
On Behalf Of Jay K

Sent: Thursday, January 23, 2014 2:57 PM

To: Tony; Coleburn, Randy

Cc: m3devel

Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?


 


There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.

Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.



 - Jay





From:
hosking at cs.purdue.edu

Date: Thu, 23 Jan 2014 11:51:02 -0500

To: rcolebur at SCIRES.COM

CC: m3devel at elegosoft.com; 
jay.krell at cornell.edu

Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?



Yes, agreed.

We need to address the native thread problem on all platforms (both pthread and windows).


 


However, this particular problem is pernicious if we need to save M3 stack state at all external calls.


 


Are there any linker tricks for Windows one can use to wrap only system calls?


 


I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,
  This approach is used for all modern Java implementations that I am aware of.


 


It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread
 state.


 




Antony Hosking | Associate
 Professor | Computer Science | Purdue University


305 N. University Street | West Lafayette | IN 47907 | USA


Mobile +1
 765 427 5484


 














 


On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:


 


Jay,


 


If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 


 


Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both
 32 & 64 bit platforms. 


 


Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 


 


Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?


 


--Randy



Sent from my iPhone





On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:



  There is a behavior/bug in wow64.  

  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.







  SuspendThread / GetThreadContext work like this:  





  32bit processes consist almost entirely of 32bit code.  

  There is a small amount of 64bit code.  





  If you suspend while running 32bit code, GetThreadContext works.  

  if you suspend while running 64bit code, GetThreadContext usually but not always works.  





  64bit code is run en route to syscalls.  

  For example you call:  

    1 kernel32!Sleep   

    2 it calls 32bit NtDelayExecution   

    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 

    4 which calls native NtDelayExecuation   



 

  In between 2 and 3, within 64bit code, the 32bit context is saved.  

    You can step through it very easily in a debugger. Really. 

  Where GetThreadContext knows where to get it.  

  The problem is that saving context is not atomic.  

  You can suspend while saving context.  





  What to do? 





  scratch/wow64stack contains a program that detects the bug.

  I believe it is the basis of a workaround for the bug.   





  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 

  not only functions that use exception handling, but also functions that call "extern" functions  

  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection

  use that, if it isn't zero. 

  Normally it will be zero.  

  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  

  That is why it isn't a set/set-zero pattern (like in the test program).   





  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 





  Rejected counter proposal: 

   Don't suspend/gc when running a syscall.  

   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  





 Possible augmentation: if running native, short circuit most of this 





  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 

   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 





   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 

   until/unless there is another 32bit platform runable on some similar 64bit platforms. 





  Performance impact: hypothetically large but probably not noticable. 





  Furthe refinements: It isn't extern/native code per se, it is syscalls.

  We could further augment pragmas to discern them. 





  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 

  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 

  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 







  Agreed? I'll make the compiler change? 





Oh, also, not just stack pointer, but other registers, at least non-volatiles?





Eventually cooperative suspend will cause this to fall away as a problem..





 - Jay





 



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140123/5aed95c6/attachment-0002.html>

From adacore at marino.st  Fri Jan 24 00:42:01 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 00:42:01 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52DBCDBA.8030809@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>
Message-ID: <52E1A8C9.3010903@marino.st>

On 1/19/2014 14:06, John Marino wrote:
> On 1/19/2014 13:50, Jay K wrote:
>> And there are advantages not yet implemented, in particular, efficient
>> exception handling by generating C++.
> 
> Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> dl_iterate_phdr which gcc uses for efficient zero-cost exception
> handling.  That might come in handy here (I implemented it in
> DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> 
> 
>> Your patches should apply fairly cleanly to any new checkout.
>> I am willing to through them and apply them to CVS if you want --
>> depends on if you are ok with my stealing your credit, vs. you want to
>> commit them yourselves.
> 
> I would love it if you did this - I'm happy with a mention in the commit
> message.  I just attached a tarball of all the patches I'm using right
> now.  Some of these changes are specifically for the cross compiler (I'm
> think of the /scripts changes mostly) but most are valid assuming they
> still apply to the head of the repo.  Please take as much as you want
> with my gratitude.
> 
>> I already skimmed them and they all look easy.
>> DragonflyBSD is basically FreeBSD by another name.
> 
> That's not an accident.  As a DragonFly developer that mostly works on
> userland, I've been converging FreeBSD and DragonFly again.  They
> drifted apart after 10 years, but I thought it better to keep the
> userlands as similar as possible for 3rd party applications.
> 
>> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
>> But the ABI is almost the same -- heck, for the most part,
>> Linux==NetBSD==FreeBSD==OpenBSD.
>> Sure, they might be implemented differently, but they are highly source
>> compatible and every highly object code compatible.
>>> I must be misunderstanding what you mean by "upgrade". It's the
>>> cross-compiler that's complaining, and it was built by the latest patches.
>> Agreed, we are both missing something simple.
>> Usually this is the error you get when you don't upgrade
>> Target.i3/Target.m3.
>> If you want to press on with the current approach and ignore my advise,
> 
> I'm not ignoring it; it's more like I'm not quite ready to give up since
> I'm so close.  I think I could actually get these libs and cm3 to build
> on DragonFly if I used AMD64_FREEBSD as the target with the
> cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> I'd then get stuck on the DragonFly side trying to compile cm3 with
> itself, theoretically I should get the same error there.
> 
> 
>> Ugh. I saw something about that in your diffs. Definitely we intend to
>> link statically.
>> I actually went to the trouble of removing use of gmp/mpfr/mpc in
>> current source.
>> They are used for actually very little and aren't worth it.
> 

Hi Jay,
If you get those patches committed and publish an official snapshot
(compressed tarball) with those changes in it, then I'll update the
FreeBSD port to that snapshot.

If new patches are required, say to convert FreeBSD to use the C
backend, I'll feed those back too.

Then I'll try porting DragonFly again, your way.
Thanks,
John





From jay.krell at cornell.edu  Fri Jan 24 09:45:39 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 08:45:39 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E1A8C9.3010903@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
Message-ID: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>

sorry, I'm delayed..so much to do..
I commited the networking fix..
The m3core/unix one I think we already have.

Later...
 - Jay

> Date: Fri, 24 Jan 2014 00:42:01 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/19/2014 14:06, John Marino wrote:
> > On 1/19/2014 13:50, Jay K wrote:
> >> And there are advantages not yet implemented, in particular, efficient
> >> exception handling by generating C++.
> > 
> > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented
> > dl_iterate_phdr which gcc uses for efficient zero-cost exception
> > handling.  That might come in handy here (I implemented it in
> > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.)
> > 
> > 
> >> Your patches should apply fairly cleanly to any new checkout.
> >> I am willing to through them and apply them to CVS if you want --
> >> depends on if you are ok with my stealing your credit, vs. you want to
> >> commit them yourselves.
> > 
> > I would love it if you did this - I'm happy with a mention in the commit
> > message.  I just attached a tarball of all the patches I'm using right
> > now.  Some of these changes are specifically for the cross compiler (I'm
> > think of the /scripts changes mostly) but most are valid assuming they
> > still apply to the head of the repo.  Please take as much as you want
> > with my gratitude.
> > 
> >> I already skimmed them and they all look easy.
> >> DragonflyBSD is basically FreeBSD by another name.
> > 
> > That's not an accident.  As a DragonFly developer that mostly works on
> > userland, I've been converging FreeBSD and DragonFly again.  They
> > drifted apart after 10 years, but I thought it better to keep the
> > userlands as similar as possible for 3rd party applications.
> > 
> >> Yes, a ton of work has been done on the kernel, file systems, maybe ports.
> >> But the ABI is almost the same -- heck, for the most part,
> >> Linux==NetBSD==FreeBSD==OpenBSD.
> >> Sure, they might be implemented differently, but they are highly source
> >> compatible and every highly object code compatible.
> >>> I must be misunderstanding what you mean by "upgrade". It's the
> >>> cross-compiler that's complaining, and it was built by the latest patches.
> >> Agreed, we are both missing something simple.
> >> Usually this is the error you get when you don't upgrade
> >> Target.i3/Target.m3.
> >> If you want to press on with the current approach and ignore my advise,
> > 
> > I'm not ignoring it; it's more like I'm not quite ready to give up since
> > I'm so close.  I think I could actually get these libs and cm3 to build
> > on DragonFly if I used AMD64_FREEBSD as the target with the
> > cm3cg-AMD_DRAGONFLY generator.  It was building before.  I just figured
> > I'd then get stuck on the DragonFly side trying to compile cm3 with
> > itself, theoretically I should get the same error there.
> > 
> > 
> >> Ugh. I saw something about that in your diffs. Definitely we intend to
> >> link statically.
> >> I actually went to the trouble of removing use of gmp/mpfr/mpc in
> >> current source.
> >> They are used for actually very little and aren't worth it.
> > 
> 
> Hi Jay,
> If you get those patches committed and publish an official snapshot
> (compressed tarball) with those changes in it, then I'll update the
> FreeBSD port to that snapshot.
> 
> If new patches are required, say to convert FreeBSD to use the C
> backend, I'll feed those back too.
> 
> Then I'll try porting DragonFly again, your way.
> Thanks,
> John
> 
> 
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9f8f3f93/attachment-0002.html>

From adacore at marino.st  Fri Jan 24 10:11:02 2014
From: adacore at marino.st (John Marino)
Date: Fri, 24 Jan 2014 10:11:02 +0100
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>, <52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>
Message-ID: <52E22E26.10402@marino.st>

On 1/24/2014 09:45, Jay K wrote:
> sorry, I'm delayed..so much to do..
> I commited the networking fix..
> The m3core/unix one I think we already have.
> 
> Later...
>  - Jay


Right, but there's no downloadable source snapshot.  And if the
DragonFly support isn't in there, I'll have to carry all those patches
too.  So my goal is to have a few patches as possible.

Anyway, I have to base the port on something -- pulling latest SVN is
rarely done, and CVS is never done.  Some github projects don't make
releases and needed to be pulled directly, but this requires first git
to be built (lots of dependencies) and then github tweaks their service
and everything fails.  So I'm a big believer in static compressed
tarballs.

Which means I can't update FreeBSD cm3 version until one exists.
theoretically I can package one myself, but it looks better if it's
"official" and comes from one of the known m3 websites.

Thanks,
John


From jay.krell at cornell.edu  Fri Jan 24 10:59:24 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 09:59:24 +0000
Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
In-Reply-To: <52E22E26.10402@marino.st>
References: <52DBA2EF.4020101@marino.st>,
	<COL130-W72F4BE524C6D7E74C39A4E6BA0@phx.gbl>
	<COL130-W4F92C4719316EF989D33BE6BA0@phx.gbl>,
	<52DBC081.1070401@marino.st>
	<COL130-W908A4361EE0D49DBB5BC77E6BA0@phx.gbl>,
	<52DBC748.5090308@marino.st>
	<COL130-W44D168BED7A76C73A2564EE6BA0@phx.gbl>
	<52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st>
	<COL130-W89906F6E3D46FC722611B9E6A10@phx.gbl>,
	<52E22E26.10402@marino.st>
Message-ID: <COL130-W58042B8075CEF3E2C8165CE6A10@phx.gbl>

Please be patient. We'll get a snapshot, soon. A release, unclear.

  - Jay

> Date: Fri, 24 Jan 2014 10:11:02 +0100
> From: adacore at marino.st
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
> 
> On 1/24/2014 09:45, Jay K wrote:
> > sorry, I'm delayed..so much to do..
> > I commited the networking fix..
> > The m3core/unix one I think we already have.
> > 
> > Later...
> >  - Jay
> 
> 
> Right, but there's no downloadable source snapshot.  And if the
> DragonFly support isn't in there, I'll have to carry all those patches
> too.  So my goal is to have a few patches as possible.
> 
> Anyway, I have to base the port on something -- pulling latest SVN is
> rarely done, and CVS is never done.  Some github projects don't make
> releases and needed to be pulled directly, but this requires first git
> to be built (lots of dependencies) and then github tweaks their service
> and everything fails.  So I'm a big believer in static compressed
> tarballs.
> 
> Which means I can't update FreeBSD cm3 version until one exists.
> theoretically I can package one myself, but it looks better if it's
> "official" and comes from one of the known m3 websites.
> 
> Thanks,
> John
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/9ee26689/attachment-0002.html>

From jay.krell at cornell.edu  Fri Jan 24 11:38:01 2014
From: jay.krell at cornell.edu (Jay K)
Date: Fri, 24 Jan 2014 10:38:01 +0000
Subject: [M3devel] garbage collector compacting and volatile?
Message-ID: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>

Is our garbage collector compacting?


In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?
Update references anywhere? Just the stack? Also context?


There are tradeoffs either way.


If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?


If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.


If we read and write registers in the collector, then nothing has to be volatile.


Should this all be parameters passed to the backend?
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.


If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.


Thank you,
 - Jay

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/e86b30bc/attachment-0002.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:24:35 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:24:35 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
Message-ID: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.
Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Mobile +1 765 427 5484





On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:

> Yes. Agreed.
> They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
> I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
>  
>  
>  - Jay
> 
>  
> From: rcolebur at SCIRES.COM
> To: m3devel at elegosoft.com
> Date: Thu, 23 Jan 2014 22:23:43 +0000
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> Jay:
>  
> When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. 
>  
> Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct?
>  
> --Randy
>  
> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
> Sent: Thursday, January 23, 2014 2:57 PM
> To: Tony; Coleburn, Randy
> Cc: m3devel
> Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
>  
> There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
> Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.
> 
>  - Jay
> From: hosking at cs.purdue.edu
> Date: Thu, 23 Jan 2014 11:51:02 -0500
> To: rcolebur at SCIRES.COM
> CC: m3devel at elegosoft.com; jay.krell at cornell.edu
> Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?
> 
> Yes, agreed.
> We need to address the native thread problem on all platforms (both pthread and windows).
>  
> However, this particular problem is pernicious if we need to save M3 stack state at all external calls.
>  
> Are there any linker tricks for Windows one can use to wrap only system calls?
>  
> I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of.
>  
> It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.
>  
> Antony Hosking | Associate Professor | Computer Science | Purdue University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Mobile +1 765 427 5484
>  
> 
> 
> 
> 
>  
> On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote:
>  
> Jay,
>  
> If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. 
>  
> Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. 
>  
> Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. 
>  
> Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?
>  
> --Randy
> 
> Sent from my iPhone
> 
> On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:
>   There is a behavior/bug in wow64.  
>   Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.
> 
> 
> 
>   SuspendThread / GetThreadContext work like this:  
> 
> 
>   32bit processes consist almost entirely of 32bit code.  
>   There is a small amount of 64bit code.  
> 
> 
>   If you suspend while running 32bit code, GetThreadContext works.  
>   if you suspend while running 64bit code, GetThreadContext usually but not always works.  
> 
> 
>   64bit code is run en route to syscalls.  
>   For example you call:  
>     1 kernel32!Sleep   
>     2 it calls 32bit NtDelayExecution   
>     3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
>     4 which calls native NtDelayExecuation   
> 
>  
>   In between 2 and 3, within 64bit code, the 32bit context is saved.  
>     You can step through it very easily in a debugger. Really. 
>   Where GetThreadContext knows where to get it.  
>   The problem is that saving context is not atomic.  
>   You can suspend while saving context.  
> 
> 
>   What to do? 
> 
> 
>   scratch/wow64stack contains a program that detects the bug.
>   I believe it is the basis of a workaround for the bug.   
> 
> 
>   Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
>   not only functions that use exception handling, but also functions that call "extern" functions  
>   call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
>   use that, if it isn't zero. 
>   Normally it will be zero.  
>   Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
>   That is why it isn't a set/set-zero pattern (like in the test program).   
> 
> 
>   Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 
> 
> 
>   Rejected counter proposal: 
>    Don't suspend/gc when running a syscall.  
>    No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  
> 
> 
>  Possible augmentation: if running native, short circuit most of this 
> 
> 
>   Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
>    compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 
> 
> 
>    AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
>    until/unless there is another 32bit platform runable on some similar 64bit platforms. 
> 
> 
>   Performance impact: hypothetically large but probably not noticable. 
> 
> 
>   Furthe refinements: It isn't extern/native code per se, it is syscalls.
>   We could further augment pragmas to discern them. 
> 
> 
>   We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
>   to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
>   It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 
> 
> 
> 
>   Agreed? I'll make the compiler change? 
> 
> 
> Oh, also, not just stack pointer, but other registers, at least non-volatiles?
> 
> 
> Eventually cooperative suspend will cause this to fall away as a problem..
> 
> 
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/df5802c0/attachment-0002.html>

From hosking at cs.purdue.edu  Fri Jan 24 19:12:33 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Fri, 24 Jan 2014 13:12:33 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>
Message-ID: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:

> Is our garbage collector compacting?

Yes, it is mostly-copying.
Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).

> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?

It reads both registers and stack.

> Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.
Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.

> There are tradeoffs either way.

Indeed. Too many to enumerate here.

> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).

> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.

No updates to values on the stacks.

> If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.

> Should this all be parameters passed to the backend?

Nope.

> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.

Nope.

> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.


It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140124/12b01775/attachment-0002.html>

From jay.krell at cornell.edu  Sat Jan 25 05:04:59 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sat, 25 Jan 2014 04:04:59 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
Message-ID: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>

I might yet have another solution here..there might be a way to detect the race and retry...later..

 - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:24:35 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context.  At least that should be the default.Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma.  Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484


On Jan 23, 2014, at 6:09 PM, Jay K <jay.krell at cornell.edu> wrote:Yes. Agreed.
They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.
I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.
 
 
 - Jay

 
From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Thu, 23 Jan 2014 22:23:43 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?

Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified.  Both of these native implementations still have threading misbehaviors as identified by the thread test program.  Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Thursday, January 23, 2014 2:57 PM
To: Tony; Coleburn, Randy
Cc: m3devel
Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.

 - JayFrom: hosking at cs.purdue.edu
Date: Thu, 23 Jan 2014 11:51:02 -0500
To: rcolebur at SCIRES.COM
CC: m3devel at elegosoft.com; jay.krell at cornell.edu
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?

Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,  This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 



 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <rcolebur at SCIRES.COM> wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64.  Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms.  Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit.  Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy

Sent from my iPhone
On Jan 23, 2014, at 2:58 AM, "Jay K" <jay.krell at cornell.edu> wrote:  There is a behavior/bug in wow64.  
  Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.



  SuspendThread / GetThreadContext work like this:  


  32bit processes consist almost entirely of 32bit code.  
  There is a small amount of 64bit code.  


  If you suspend while running 32bit code, GetThreadContext works.  
  if you suspend while running 64bit code, GetThreadContext usually but not always works.  


  64bit code is run en route to syscalls.  
  For example you call:  
    1 kernel32!Sleep   
    2 it calls 32bit NtDelayExecution   
    3 that calls wow64NtDelayExecutation  (via a cross segment "far" jmp or call) 
    4 which calls native NtDelayExecuation   

 
  In between 2 and 3, within 64bit code, the 32bit context is saved.  
    You can step through it very easily in a debugger. Really. 
  Where GetThreadContext knows where to get it.  
  The problem is that saving context is not atomic.  
  You can suspend while saving context.  


  What to do? 


  scratch/wow64stack contains a program that detects the bug.
  I believe it is the basis of a workaround for the bug.   


  Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, 
  not only functions that use exception handling, but also functions that call "extern" functions  
  call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection
  use that, if it isn't zero. 
  Normally it will be zero.  
  Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile.  
  That is why it isn't a set/set-zero pattern (like in the test program).   


  Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. 


  Rejected counter proposal: 
   Don't suspend/gc when running a syscall.  
   No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that.  


 Possible augmentation: if running native, short circuit most of this 


  Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate 
   compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. 


   AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, 
   until/unless there is another 32bit platform runable on some similar 64bit platforms. 


  Performance impact: hypothetically large but probably not noticable. 


  Furthe refinements: It isn't extern/native code per se, it is syscalls.
  We could further augment pragmas to discern them. 


  We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls 
  to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. 
  It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... 



  Agreed? I'll make the compiler change? 


Oh, also, not just stack pointer, but other registers, at least non-volatiles?


Eventually cooperative suspend will cause this to fall away as a problem..


 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140125/b19a595a/attachment-0002.html>

From hendrik at topoi.pooq.com  Sat Jan 25 05:38:03 2014
From: hendrik at topoi.pooq.com (Hendrik Boom)
Date: Fri, 24 Jan 2014 23:38:03 -0500
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>
Message-ID: <20140125043803.GA3423@topoi.pooq.com>

On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> I might yet have another solution here..there might be a way to 
> detect the race and retry...later..
> 
>  - Jay

I seem to remember a lot of comments in the trestle code tht seem to 
be intended as assertions to catch race conditions.
Is any of that stuff conceivably helpful here?

-- hendrik


From jay.krell at cornell.edu  Sun Jan 26 07:08:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <20140125043803.GA3423@topoi.pooq.com>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	<COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>,
	<20140125043803.GA3423@topoi.pooq.com>
Message-ID: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>

No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140126/e01d9d37/attachment-0002.html>

From jay.krell at cornell.edu  Mon Jan 27 08:23:26 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:23:26 +0000
Subject: [M3devel] proposal for getting wow64 context/stack reliably?
In-Reply-To: <COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>,
	, <COL130-W5716F6F39E5BEDC7B33CADE6A60@phx.gbl>, ,
	<D0E24A22-0865-4ECC-8575-4F883FC7D527@cs.purdue.edu>, ,
	<COL130-W7976F330BFD72741E74493E6A00@phx.gbl>, ,
	<20140125043803.GA3423@topoi.pooq.com>,
	<COL130-W67BD9C628B17840E9719B4E6A30@phx.gbl>
Message-ID: <COL130-W141C96275EF6018B1D2D0E6A20@phx.gbl>


 > I might yet have another solution here
And it might only be for Windows 8 or newer.I tend to think we should go with the earlier proposal -- save context ourselves in thread locals when calling externals. Likely portable implementation is setjmp or getcontext, though we can maybe do better.This would be part of cooperative suspend anyway, which we want to move to.

 - Jay

From: jay.krell at cornell.edu
To: hendrik at topoi.pooq.com; m3devel at elegosoft.com
Date: Sun, 26 Jan 2014 06:08:20 +0000
Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?




No.



 
> Date: Fri, 24 Jan 2014 23:38:03 -0500
> From: hendrik at topoi.pooq.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?
> 
> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote:
> > I might yet have another solution here..there might be a way to 
> > detect the race and retry...later..
> > 
> >  - Jay
> 
> I seem to remember a lot of comments in the trestle code tht seem to 
> be intended as assertions to catch race conditions.
> Is any of that stuff conceivably helpful here?
> 
> -- hendrik
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/146c0737/attachment-0002.html>

From jay.krell at cornell.edu  Mon Jan 27 08:45:08 2014
From: jay.krell at cornell.edu (Jay K)
Date: Mon, 27 Jan 2014 07:45:08 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
Message-ID: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>

In summary, shortest:

  We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).    We must be able to read the stack.    We need not write registers or stack (from garbage collector).  

Slight elaboration: Copying/compaction only occurs when references are only from globals and heap.  Therefore updates/writes only occur to globals and heap. 

 Is there any desire to do better/different?  To require the ability to update stack/registers?  What does Java typically to? 


I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.To do better/different would require more backend cooperation.We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Thanks, - Jay

From: hosking at cs.purdue.edu
Date: Fri, 24 Jan 2014 13:12:33 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] garbage collector compacting and volatile?

On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
Is our garbage collector compacting?

Yes, it is mostly-copying.Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
In particular, what model do we need for "volatile" in codegen?
Does the compactor read the registers or just the stack?

It reads both registers and stack.
Update references anywhere? Just the stack? Also context?

The collector first pins all pages referenced from the stacks, including registers.Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
There are tradeoffs either way.

Indeed. Too many to enumerate here.
If it is not compacting and only reads the stack, then it can be more
portable, but every write of a collected type/pointer would have to be volatile.
Or is that what the barriers are for?

The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
If we update values on the stack, then there is some portability,
but reads and writes would have to be volatile.

No updates to values on the stacks.
If we read and write registers in the collector, then nothing has to be volatile.

Read only.  No writes to registers.
Should this all be parameters passed to the backend?

Nope.
And/or preserved via #ifdef in the C backend?
That last part is easy enough, for generated C to take #ifdefs
indicating if reads and/or writes should be volatile.

Nope.
If the collector is not compacting, then it need not update
stack or register values, just possibly read them.

I recall reading that it is compacting.

It is mostly-copying.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/9dd883b4/attachment-0002.html>

From hosking at cs.purdue.edu  Mon Jan 27 16:16:57 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 27 Jan 2014 10:16:57 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
Message-ID: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>



On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:

> In summary, shortest:
> 
> 
>   We must be able to read registers (or save them to the stack or elsewhere (known thread locals)).  
>   We must be able to read the stack.  
>   We need not write registers or stack (from garbage collector).  

Correct.

> Slight elaboration:
>  Copying/compaction only occurs when references are only from globals and heap. 
>  Therefore updates/writes only occur to globals and heap. 

Correct.

>  Is there any desire to do better/different? 
>  To require the ability to update stack/registers? 

Without control of the register allocator, optimisations, and stack layout performed by the compiler (especially back-end) getting fully accurate stack maps is very difficult.

>  What does Java typically to? 

Java VMs usually control their own destiny, having full control of JIT and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.

> I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.

Correct.

> Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.

Correct.

> To do better/different would require more backend cooperation.
> We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them.

Correct.

> 
> 
> Thanks,
>  - Jay
> 
> From: hosking at cs.purdue.edu
> Date: Fri, 24 Jan 2014 13:12:33 -0500
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> On Jan 24, 2014, at 5:38 AM, Jay K <jay.krell at cornell.edu> wrote:
> 
> Is our garbage collector compacting?
> 
> Yes, it is mostly-copying.
> Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted).
> 
> In particular, what model do we need for "volatile" in codegen?
> Does the compactor read the registers or just the stack?
> 
> It reads both registers and stack.
> 
> Update references anywhere? Just the stack? Also context?
> 
> The collector first pins all pages referenced from the stacks, including registers.
> Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location.
> 
> There are tradeoffs either way.
> 
> Indeed. Too many to enumerate here.
> 
> If it is not compacting and only reads the stack, then it can be more
> portable, but every write of a collected type/pointer would have to be volatile.
> Or is that what the barriers are for?
> 
> The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled).
> 
> If we update values on the stack, then there is some portability,
> but reads and writes would have to be volatile.
> 
> No updates to values on the stacks.
> 
> If we read and write registers in the collector, then nothing has to be volatile.
> 
> Read only.  No writes to registers.
> 
> Should this all be parameters passed to the backend?
> 
> Nope.
> 
> And/or preserved via #ifdef in the C backend?
> That last part is easy enough, for generated C to take #ifdefs
> indicating if reads and/or writes should be volatile.
> 
> Nope.
> 
> If the collector is not compacting, then it need not update
> stack or register values, just possibly read them.
> 
> I recall reading that it is compacting.
> 
> It is mostly-copying.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/6f9b0cfd/attachment-0002.html>

From rodney_bates at lcwb.coop  Mon Jan 27 20:46:23 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 13:46:23 -0600
Subject: [M3devel] New compile failure
Message-ID: <52E6B78F.7000003@lcwb.coop>

The compiler in the head, with some recent changes, gives:

rodney at yellowstone:~/proj/m3/cm3-head/cm3/m3-sys/m3middle$ cm3

Fatal Error: configuration file didn't specify BUILD_DIR

This message is coming from cm3/src/Main.m3, which has recently
changed, but there are related changes to other source files
too.  Reverting Main.m3 to the previous version won't compile.

Putting an older executable for cm3 into the bin directory,
with recently updated and installed cm3.cfg and config works.


From rodney_bates at lcwb.coop  Mon Jan 27 21:49:43 2014
From: rodney_bates at lcwb.coop (Rodney M. Bates)
Date: Mon, 27 Jan 2014 14:49:43 -0600
Subject: [M3devel] narrow still failing..
In-Reply-To: <52E13487.9090504@lcwb.coop>
References: <20140119213649.403AE1A208F@async.async.caltech.edu>
	<20140119220123.0D1971A208F@async.async.caltech.edu>
	<52E0202F.3020307@lcwb.coop>
	<20140122204749.C724E1A208F@async.async.caltech.edu>
	<52E13487.9090504@lcwb.coop>
Message-ID: <52E6C667.6030708@lcwb.coop>

It should be fixed in the head, now.

On 01/23/2014 09:25 AM, Rodney M. Bates wrote:
>
>
> On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote:
>> Aha so declaring the type as <: ROOT will probably work as a workaround?
>>
>
> I'm not sure whether that would do it or not.  In my pared-down case, I had
> moved BDD.T and BDD.Root into main, but left the REFANY variables.  That made
> the failures go away.
>
> Until it is fixed, I think the best workaround is TYPECASE with a binding:
>
> TYPECASE x
> OF BDD.T(b) =>
>    Format(b), etc.
>
> This is known to be working now.  You've probably thought about this, but
> this way requires only a single runtime type test.  Either TYPECASE without
> the binding of b, or ISTYPE, will require an additional recheck of the same
> thing when the narrow, implicit or explicit, happens.  I doubt even a
> pretty good optimizing compiler could remove this.  For ISTYPE, it would have
> to know that the runtime routine (e.g. CheckIsType) was a pure function.  Pretty
> unlikely.  Even worse for TYPECASE, where it would have to know a lot about the
> relation between CheckIsType and ScanTypecase.
>
>> Thank you for looking into this.
>>
>> "Rodney M. Bates" writes:
>>> Some more info:
>>>
>>> The front end is generating incorrect narrow code, only
>>> where the type T is an unrevealed <: REFANY.  When T and Root
>>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW,
>>> and implicit narrow in an assignment.
>>>
>>> When opaque, for explicit and implicit narrow, it simply compares
>>> the typecode of x to a single typecode (no doubt for T) and demands
>>> they be equal.  In this case, the actual allocated type of x is Root,
>>> a proper subtype of T.
>>>
>>> For ISTYPE, it does call CheckIsType even when the types are opaque.
>>>
>>> The fact that it happens only when the types are opaque lends
>>> credibility to the fact that this bug has gone unnoticed at least
>>> as far back as PM3.
>>
>
>



From jay.krell at cornell.edu  Tue Jan 28 00:13:46 2014
From: jay.krell at cornell.edu (Jay)
Date: Mon, 27 Jan 2014 15:13:46 -0800
Subject: [M3devel] Fwd: Why does std::chrono now() uses slow syscall?
References: <CAH6eHdSjowHv+AM=T7nKgyoVuthbg6dAVbBvP0+UBxg0w=_jgg@mail.gmail.com>
Message-ID: <9FD55748-9752-4E40-9605-78F98AE62A5D@gmail.com>

Hints for how m3core/libm3 should get time efficiently...

 - Jay

Begin forwarded message:

> From: Jonathan Wakely <jwakely.gcc at gmail.com>
> Date: January 26, 2014, 2:33:36 AM PST
> To: Keith Erickson <kerickso at pppl.gov>
> Cc: gcc-help <gcc-help at gcc.gnu.org>
> Subject: Re: Why does std::chrono now() uses slow syscall?
> 
> On 26 January 2014 07:50, Keith Erickson wrote:
>> My questions, then, are....  why would I ever want to use the syscall
>> version?  It seems that configure prefers it, and will use it if it's
>> available.
> 
> It prefers to use clock_gettime and only uses the syscall if a
> configure test using clock_gettime(CLOCK_MONOTONIC, &tp) fails.
> 
>> But it's stupidly slow, even on the fasted server CPU that
>> AMD sells (Opteron 6386 SE at the time of this writing).  How do I not
>> use this slow method?  Do I have to compile gcc specially?  What are
>> the drawbacks to forcing configure to not set that #define?  Is there
>> an approved way to tell configure to use a fast time?
> 
> Prior to glibc 2.17 clock_gettime was in librt.so not libc.so, which
> is not used automatically by libstdc++. To tell configure to use it
> you need to build GCC with --enable-libstdcxx-time=rt, but that means
> that libstdc++.so will be linked to librt.so which depends on
> libpthread.so which means that the library always thinks it is part of
> a multithreaded application and so even single-threaded programs use
> atomic operations for internal reference-counting (e.g. in std::string
> and std::shared_ptr).  That's why we don't use clock_gettime unless
> explicitly configured to do so.
> 
> With glibc 2.17 or later clock_gettime will be used automatically,
> because the functions were moved out of librt.so.
> 
> On my Fedora 20 system I get
> 
> | #define HAVE_SYS_TIME_H 1
> | #define _GLIBCXX_USE_GETTIMEOFDAY 1
> | #define _GLIBCXX_USE_CLOCK_MONOTONIC 1
> | #define _GLIBCXX_USE_CLOCK_REALTIME 1
> | #define _GLIBCXX_USE_SCHED_YIELD 1
> | #define _GLIBCXX_USE_NANOSLEEP 1
> 
> which means it doesn't need the syscall version.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140127/d9369ca7/attachment-0002.html>

From mika at async.caltech.edu  Wed Jan 29 00:49:19 2014
From: mika at async.caltech.edu (mika at async.caltech.edu)
Date: Tue, 28 Jan 2014 15:49:19 -0800
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
Message-ID: <20140128234919.0E0C81A208F@async.async.caltech.edu>


Sounds like Jay doesn't need to buy a copy of your book :-)

Tony Hosking writes:
>
>--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>Content-Transfer-Encoding: quoted-printable
>Content-Type: text/plain;
>	charset=iso-8859-1
>
>
>
>On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>
>> In summary, shortest:
>>=20
>>=20
>>   We must be able to read registers (or save them to the stack or =
>elsewhere (known thread locals)). =20
>>   We must be able to read the stack. =20
>>   We need not write registers or stack (from garbage collector). =20
>
>Correct.
>
>> Slight elaboration:
>>  Copying/compaction only occurs when references are only from globals =
>and heap.=20
>>  Therefore updates/writes only occur to globals and heap.=20
>
>Correct.
>
>>  Is there any desire to do better/different?=20
>>  To require the ability to update stack/registers?=20
>
>Without control of the register allocator, optimisations, and stack =
>layout performed by the compiler (especially back-end) getting fully =
>accurate stack maps is very difficult.
>
>>  What does Java typically to?=20
>
>Java VMs usually control their own destiny, having full control of JIT =
>and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>
>> I guess there is the problem that we don't know for certain if values =
>in registers and stack are pointers, or just coincident integers, and so =
>updating them can be wrong.
>
>Correct.
>
>> Whereas in the globals and "reachable thereof" we have precise type =
>information for, so we can update those values safely.
>
>Correct.
>
>> To do better/different would require more backend cooperation.
>> We conservatively treat registers/stack as pointers if they look like =
>them, and therefore keep values alive, but not trust them so much as to =
>update them.
>
>Correct.


From jay.krell at cornell.edu  Wed Jan 29 01:00:20 2014
From: jay.krell at cornell.edu (Jay K)
Date: Wed, 29 Jan 2014 00:00:20 +0000
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>, ,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>,
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>,
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>,
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <COL130-W51823A67DAA7B4D7B7BDD3E6AC0@phx.gbl>

Good point.  I have it around somewhere... (I need that automatic resource management in real life..)
 
 - Jay

 
> To: hosking at cs.purdue.edu
> Date: Tue, 28 Jan 2014 15:49:19 -0800
> From: mika at async.caltech.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] garbage collector compacting and volatile?
> 
> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
> >
> >--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
> >Content-Transfer-Encoding: quoted-printable
> >Content-Type: text/plain;
> >	charset=iso-8859-1
> >
> >
> >
> >On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
> >
> >> In summary, shortest:
> >>=20
> >>=20
> >>   We must be able to read registers (or save them to the stack or =
> >elsewhere (known thread locals)). =20
> >>   We must be able to read the stack. =20
> >>   We need not write registers or stack (from garbage collector). =20
> >
> >Correct.
> >
> >> Slight elaboration:
> >>  Copying/compaction only occurs when references are only from globals =
> >and heap.=20
> >>  Therefore updates/writes only occur to globals and heap.=20
> >
> >Correct.
> >
> >>  Is there any desire to do better/different?=20
> >>  To require the ability to update stack/registers?=20
> >
> >Without control of the register allocator, optimisations, and stack =
> >layout performed by the compiler (especially back-end) getting fully =
> >accurate stack maps is very difficult.
> >
> >>  What does Java typically to?=20
> >
> >Java VMs usually control their own destiny, having full control of JIT =
> >and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
> >
> >> I guess there is the problem that we don't know for certain if values =
> >in registers and stack are pointers, or just coincident integers, and so =
> >updating them can be wrong.
> >
> >Correct.
> >
> >> Whereas in the globals and "reachable thereof" we have precise type =
> >information for, so we can update those values safely.
> >
> >Correct.
> >
> >> To do better/different would require more backend cooperation.
> >> We conservatively treat registers/stack as pointers if they look like =
> >them, and therefore keep values alive, but not trust them so much as to =
> >update them.
> >
> >Correct.
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20140129/895ab0a2/attachment-0002.html>

From hosking at cs.purdue.edu  Wed Jan 29 01:07:15 2014
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 28 Jan 2014 19:07:15 -0500
Subject: [M3devel] garbage collector compacting and volatile?
In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu>
References: <COL130-W14340EDE9DC9CE32676848E6A10@phx.gbl>,
	<8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>
	<COL130-W34EC06F2FBC1636278E96BE6A20@phx.gbl>
	<4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>
	<20140128234919.0E0C81A208F@async.async.caltech.edu>
Message-ID: <38455303-D2F4-4BDD-8E31-D7139D566436@cs.purdue.edu>

;-)

I should send him a copy!

On Jan 28, 2014, at 6:49 PM, mika at async.caltech.edu wrote:

> 
> Sounds like Jay doesn't need to buy a copy of your book :-)
> 
> Tony Hosking writes:
>> 
>> --Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/plain;
>> 	charset=iso-8859-1
>> 
>> 
>> 
>> On Jan 27, 2014, at 2:45 AM, Jay K <jay.krell at cornell.edu> wrote:
>> 
>>> In summary, shortest:
>>> =20
>>> =20
>>>  We must be able to read registers (or save them to the stack or =
>> elsewhere (known thread locals)). =20
>>>  We must be able to read the stack. =20
>>>  We need not write registers or stack (from garbage collector). =20
>> 
>> Correct.
>> 
>>> Slight elaboration:
>>> Copying/compaction only occurs when references are only from globals =
>> and heap.=20
>>> Therefore updates/writes only occur to globals and heap.=20
>> 
>> Correct.
>> 
>>> Is there any desire to do better/different?=20
>>> To require the ability to update stack/registers?=20
>> 
>> Without control of the register allocator, optimisations, and stack =
>> layout performed by the compiler (especially back-end) getting fully =
>> accurate stack maps is very difficult.
>> 
>>> What does Java typically to?=20
>> 
>> Java VMs usually control their own destiny, having full control of JIT =
>> and stack layout.  viz. HotSpot, IBM J9, Jikes RVM.
>> 
>>> I guess there is the problem that we don't know for certain if values =
>> in registers and stack are pointers, or just coincident integers, and so =
>> updating them can be wrong.
>> 
>> Correct.
>> 
>>> Whereas in the globals and "reachable thereof" we have precise type =
>> information for, so we can update those values safely.
>> 
>> Correct.
>> 
>>> To do better/different would require more backend cooperation.
>>> We conservatively treat registers/stack as pointers if they look like =
>> them, and therefore keep values alive, but not trust them so much as to =
>> update them.
>> 
>> Correct.