From dragisha at m3w.org  Wed May  2 07:45:52 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 07:45:52 +0200
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
Message-ID: <1178084752.8631.3.camel@faramir.m3w.org>

  Is it possible to get this fix separatelly from threading fixes? It's
more important, esp since only program that crashes threading is
TestThreads :)

  TIA,
  dd

On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
> Yes, this fix is in CVS head.
> 
> 
> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
> 
> > It looks like this fix never came through... With cvs-head CM3,
> > unpickling of TextRefTbl.T instance still does not work.
> >
> > dd
> >
> > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
> >> Rodney,
> >>
> >> I've checked in fixes to RTTypeMap and clients that should restore
> >> functional pickling support.  For some reason my commit messages are
> >> awaiting moderator approval (it seems my commit id is not the same as
> >> my subscription to the m3commit list) before they can be sent out.
> >> Please make sure that unpickling now works for you.
> >>
> >> Best regards,
> >>
> >> Tony
> >>
> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
> >>
> >>> I tracked down a bug unpickling to the following.  When executing
> >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
> >>>
> >>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
> >>>                   ref^ := v.readRef(nelem.refType);
> >>>                 END;
> >>>
> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref).
> >>> CheckStoreTraced expects ref to be a normal reference to an object,
> >>> computes the address of its header (4 bytes less), and sets the
> >>> dirty bit.
> >>>
> >>> But ConvertPacking is reading in to a field of a recently created
> >>> object that is not first in the object.  It computes ref to point
> >>> to the field, and CheckStoreTraced actually steps on the previous
> >>> real field instead of the header.
> >>>
> >>> It looks like the actions of CheckStoreTraced are needed, but I
> >>> don't right off hand see how to recode Convert to fix this.  It's
> >>> use of ref can't be moved.  Presumably, there could be other unsafe
> >>> code elsewhere using the same technique to store data.
> >>>
> >>> Does the compiler need a different criterion on when to generate
> >>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
> >>> still is pointing inside a traced object.   Maybe people who modify
> >>> objects in this way need to explicitly call CheckStoreTraced?
> >>>
> >>> -- 
> >>> -------------------------------------------------------------
> >>> Rodney M. Bates, retired assistant professor
> >>> Dept. of Computer Science, Wichita State University
> >>> Wichita, KS 67260-0083
> >>> 316-978-3922
> >>> rodney.bates at wichita.edu
> >>> _______________________________________________
> >>> M3devel mailing list
> >>> M3devel at elegosoft.com
> >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> >>
> >> Antony Hosking                | Associate Professor
> >> Dept of Computer Science      | Office: +1 765 494-6001
> >> Purdue University             | Mobile: +1 765 427-5484
> >> 250 N. University Street      | Email:  hosking at cs.purdue.edu
> >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
> >>   _--_|\
> >> /      \
> >> \_.--._/    )
> >>        v    /
> >>
> >>
> >>
> >> _______________________________________________
> >> M3devel mailing list
> >> M3devel at elegosoft.com
> >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May  2 08:18:22 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 2 May 2007 02:18:22 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
	<1178084752.8631.3.camel@faramir.m3w.org>
Message-ID: <F3AF7610-4655-4662-A530-ECCE833A1464@cs.purdue.edu>

Just grab CVS from before 2007/03/2007 (when I broke GC).  The  
pertinent fix was to RTTypeMap.m3, as at the current CVS head, which  
is dated 2007/01/09.

Thread/GC-related fixes are coming -- I know what the problem is but  
am in the middle of refactoring the heap allocation code.

On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote:

>   Is it possible to get this fix separatelly from threading fixes?  
> It's
> more important, esp since only program that crashes threading is
> TestThreads :)
>
>   TIA,
>   dd
>
> On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
>> Yes, this fix is in CVS head.
>>
>>
>> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
>>
>>> It looks like this fix never came through... With cvs-head CM3,
>>> unpickling of TextRefTbl.T instance still does not work.
>>>
>>> dd
>>>
>>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
>>>> Rodney,
>>>>
>>>> I've checked in fixes to RTTypeMap and clients that should restore
>>>> functional pickling support.  For some reason my commit messages  
>>>> are
>>>> awaiting moderator approval (it seems my commit id is not the  
>>>> same as
>>>> my subscription to the m3commit list) before they can be sent out.
>>>> Please make sure that unpickling now works for you.
>>>>
>>>> Best regards,
>>>>
>>>> Tony
>>>>
>>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
>>>>
>>>>> I tracked down a bug unpickling to the following.  When executing
>>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
>>>>>
>>>>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
>>>>>                   ref^ := v.readRef(nelem.refType);
>>>>>                 END;
>>>>>
>>>>> has a compiler-generated call on RTCollector.CheckStoreTraced 
>>>>> (ref).
>>>>> CheckStoreTraced expects ref to be a normal reference to an  
>>>>> object,
>>>>> computes the address of its header (4 bytes less), and sets the
>>>>> dirty bit.
>>>>>
>>>>> But ConvertPacking is reading in to a field of a recently created
>>>>> object that is not first in the object.  It computes ref to point
>>>>> to the field, and CheckStoreTraced actually steps on the previous
>>>>> real field instead of the header.
>>>>>
>>>>> It looks like the actions of CheckStoreTraced are needed, but I
>>>>> don't right off hand see how to recode Convert to fix this.  It's
>>>>> use of ref can't be moved.  Presumably, there could be other  
>>>>> unsafe
>>>>> code elsewhere using the same technique to store data.
>>>>>
>>>>> Does the compiler need a different criterion on when to generate
>>>>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
>>>>> still is pointing inside a traced object.   Maybe people who  
>>>>> modify
>>>>> objects in this way need to explicitly call CheckStoreTraced?
>>>>>
>>>>> -- 
>>>>> -------------------------------------------------------------
>>>>> Rodney M. Bates, retired assistant professor
>>>>> Dept. of Computer Science, Wichita State University
>>>>> Wichita, KS 67260-0083
>>>>> 316-978-3922
>>>>> rodney.bates at wichita.edu
>>>>> _______________________________________________
>>>>> M3devel mailing list
>>>>> M3devel at elegosoft.com
>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>> Antony Hosking                | Associate Professor
>>>> Dept of Computer Science      | Office: +1 765 494-6001
>>>> Purdue University             | Mobile: +1 765 427-5484
>>>> 250 N. University Street      | Email:  hosking at cs.purdue.edu
>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
>>>>   _--_|\
>>>> /      \
>>>> \_.--._/    )
>>>>        v    /
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From ronny.forberger at elegosoft.com  Wed May  2 11:24:14 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 02 May 2007 11:24:14 +0200
Subject: [M3devel] Just a test - please ignore
Message-ID: <463858BE.1040000@elegosoft.com>


-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From dragisha at m3w.org  Wed May  2 23:11:42 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 23:11:42 +0200
Subject: [M3devel] formals initialization... NIL?
Message-ID: <1178140302.8631.26.camel@faramir.m3w.org>

99      PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args:
ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User }
=
100       BEGIN
101         IF args[FIRST(args)] = NIL THEN
102           RAISE User(
103             NEW(RefSeq.T).fromArray(
104               ARRAY OF REFANY { e.getRef(f), e.getQName()}
105             )
(m3gdb) p args
$1 = [16_00000001]

Is this normal? 16_00000001 is NIL? Why not printed as such?

-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May  5 09:44:07 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 05 May 2007 09:44:07 +0200
Subject: [M3devel] status of -linkall?
Message-ID: <1178351048.21091.3.camel@faramir.m3w.org>

I have a situation where -linkall works "as advertised", at least I
don't see anything missing yet :). Same set of modules, when included in
build of another program, fails to initialize...

If I've let anyone forget it :), it's plugin architecture where modules
plug to appropriate "connection points" simply by inclusion in
m3makefile.

Is it known problem with -linkall?

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Mon May  7 08:03:45 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 07 May 2007 08:03:45 +0200
Subject: [M3devel] import/exports loops... dynamic loading of Modula-3
	modules
Message-ID: <1178517825.3925.23.camel@faramir.m3w.org>

I am thinking about writing some loop detecting tool for (in)famous
order-of-initialization sindrome. Anyone did it already? I think we
really need something to help sort out of EXPORTS nightmare and it's
influence on RTLinker behaviour.

Also in news :), after I've tried to solve my problems with -linkall
(and got very big executabales as consequence, together with almost
working programs - at least until I try to make another similar build :)
- dynamic loading of modules was logical next step.

I've done it, and can confirm RTLinker.AddUnit works as advertised, any
time one calls it :) It also solves my plugin architecture problem, as
my plugin modules are not imported directly and thus their
initialization is fully performed with adequate RTLinker.AddUnit call.

If there's interest in how-to-make-dynamic-modules-happen, I can post it
somewhere. My google search did not show ay Modula-3 work on that.

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Wed May  9 01:55:18 2007
From: darko at darko.org (Darko)
Date: Wed, 9 May 2007 01:55:18 +0200
Subject: [M3devel] m3.org expiry
Message-ID: <B588AC8B-D2DD-4271-B67D-4CC95C8E6F02@darko.org>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.



From wagner at elegosoft.com  Wed May  9 07:57:41 2007
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 9 May 2007 07:57:41 +0200
Subject: [M3devel] [darko@darko.org:  m3.org expiry]
Message-ID: <20070509055740.GA20162@elegosoft.com>

Hi Farshad,

I seem to remember that you tried to register m3.org for a period
longer than a year, but now it's gone again:

% host www.m3.org
www.m3.org is an alias for resalehost.networksolutions.com.
resalehost.networksolutions.com has address 205.178.189.128

What's the current state of things from your side? Should anybody
else get and host m3.org?

Olaf

----- Forwarded message from Darko <darko at darko.org> -----

X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
To: Modula-3 developers <m3devel at elegosoft.com>
From: Darko <darko at darko.org>
Date: Wed, 9 May 2007 01:55:18 +0200
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed 
	version=3.0.3
X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de
X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de
X-Virus-Status: Clean
Subject: [M3devel] m3.org expiry
X-BeenThere: m3devel at elegosoft.com
X-Mailman-Version: 2.1.9
List-Id: <m3devel.elegosoft.com>
List-Unsubscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=unsubscribe>
List-Archive: <http://mail.elegosoft.com/pipermail/m3devel>
List-Post: <mailto:m3devel at elegosoft.com>
List-Help: <mailto:m3devel-request at elegosoft.com?subject=help>
List-Subscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=subscribe>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel

----- End forwarded message -----

-- 
Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 23 45 869  fax: +49 30 23 45 86 95
http://www.elegosoft.com  |  Gesch?ftsf?hrer: Olaf Wagner  |   Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194


From hosking at cs.purdue.edu  Wed May 16 17:21:12 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:21:12 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <A79828E1-0986-4552-B0A1-C93811547613@cs.purdue.edu>

TestThreads is still locking up with 3000 threads (it works with 2000  
on my I386_DARWIN box).  I am now tracking this down.

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May 16 17:19:54 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:19:54 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org>
References: <1177277134.737.47.camel@faramir.m3w.org>
	<9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu>
	<19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu>
	<1177281325.737.55.camel@faramir.m3w.org>
	<F32E2364-A890-4739-AC5F-D1E46616C503@cs.purdue.edu>
	<1177308751.737.63.camel@faramir.m3w.org>
Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu>

On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote:

> Thank you. I am using your posted compiler for now.
>
> dd
>
> On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote:
>> Something appears to be very broken with my most recent fixes for
>> your threading problems (since Apr 19, when I put the last cm3.gz
>> compiler bootstrap up).  I will try to track this down tomorrow.

I have tracked down and fixed this brokenness.  CVS head should now  
behave reasonably.

>>
>> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote:
>>
>>> Any pointer on how to do this?
>>>
>>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am
>>> using.
>>>
>>> dd
>>>
>>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote:
>>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt
>>>> for this fix to take effect.
>>>>
>>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote:
>>>>
>>>>> This works just fine for me on the most recent CVS head for
>>>>> I386_DARWIN.
>>>>>
>>>>> I wonder if you have not updated your compiler to match?  There  
>>>>> was
>>>>> a matching fix to the compiler.  You should build and ship:
>>>>>
>>>>> m3gc-simple
>>>>> m3core
>>>>> libm3
>>>>> m3middle
>>>>> m3linker
>>>>> m3front
>>>>> m3quake
>>>>> cm3
>>>>>
>>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your
>>>>> cm3.cfg).  This will get you a new compiler.
>>>>>
>>>>> Then, rebuild your CM3 installation from the CVS head.
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: Dragi?a Duri? <dragisha at m3w.org>
>>>>>> Date: April 22, 2007 5:25:33 PM EDT
>>>>>> To: Tony Hosking <hosking at cs.purdue.edu>
>>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug
>>>>>>
>>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote:
>>>>>>> Can you send me a stack dump from gdb at the point of the error?
>>>>>>> Even better, do you have a simple example program I can use to
>>>>>>> test
>>>>>>> with?
>>>>>>
>>>>>> Here is one. Quick and dirty, but it shows problem.
>>>>>>
>>>>>> -- 
>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>> <Pickles.m3>
>>>>>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Fri May 18 13:50:24 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Fri, 18 May 2007 13:50:24 +0200
Subject: [M3devel] Pickle compatibility...
Message-ID: <1179489025.6698.3.camel@faramir.m3w.org>

I have some data pickled by pm3 pickler... Now, reading these fails...
First app I've tried had online quiz high scores pickled, but now I have
to convert billing app :). What can I do to convert this?

dd

-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Sat May 19 22:00:36 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 19 May 2007 16:00:36 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>

I just checked in a patch that is closer to your code, and that lets  
me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
creation fails for that many).

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May 19 22:16:47 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 19 May 2007 22:16:47 +0200
Subject: [M3devel] pthread in cm3...
In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
	<0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
Message-ID: <1179605808.8468.4.camel@faramir.m3w.org>

I am running 4000, but it's not most important anyway...

Did you see that (Un)LockMutex semantics I've enabled so one can
overload methods invoked by LOCK .. DO?

dd

On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote:
> I just checked in a patch that is closer to your code, and that lets  
> me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
> creation fails for that many).
> 
> On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:
> 
> > After implementing that workaround for result code 11 in that  
> > SIGSUSPEND
> > loop, every time during 1st or at most second pass (of 4000) it  
> > stucks.
> > Not same place every time, though... I think there are RCs in
> > LockHeap/SuspendAll logic.
> >
> > dd
> >
> > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
> >> Yes, good thinking.  Tuning the threads systems is a good plan all
> >> round.
> >>
> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
> >>
> >>> Just a random thought...
> >>>
> >>> I don't think my TestThreads is something special, but it's few  
> >>> thread
> >>> use patterns combined... And I've just had bright :) idea  
> >>> yesterday -
> >>> it's also decent benchmark for whole threading system... I think it
> >>> would be nice to test it with 10000 rounds, 4000 threads each (once
> >>> cm3
> >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
> >>> tests for
> >>> mine.
> >>>
> >>> I think these extra data structures and global locks in PTHREAD are
> >>> big
> >>> efficiency killers. Benchmark will show.
> >>>
> >>> dd
> >>>
> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
> >>>> I take all of your points seriously.  One option would be to offer
> >>>> your threads implementation as another build option for CM3.  I'm
> >>>> going to track down the bug I introduced recently and then we can
> >>>> consider how to move forward.
> >>>>
> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
> >>>>
> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
> >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
> >>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> Have been skimming through source of PTHREAD code... And I think
> >>>>>>> job can
> >>>>>>> be done without so much relying on how-they-did-it-before, esp
> >>>>>>> with
> >>>>>>> regard to list of waiters and similar internal and global
> >>>>>>> structures.
> >>>>>>> Also, I see number of global locks and I am sure they are
> >>>>>>> congestion
> >>>>>>> generators every now and while, esp in heavy threading  
> >>>>>>> situations.
> >>>>>>>
> >>>>>>> Of course, there is number of approaches to this multi-thread
> >>>>>>> situations. Mine being one of very nonconservative use of
> >>>>>>> threads, I
> >>>>>>> think it is important to remain open to possibly very big
> >>>>>>> number of
> >>>>>>> threads running in single process - meaning scalability is  
> >>>>>>> one of
> >>>>>>> primary objections... As global locks don't do well with
> >>>>>>> scalability, I
> >>>>>>> don't like "cm" and similar global congestion points.
> >>>>>>
> >>>>>> Yes, there are tensions between a thin/absent veneer between
> >>>>>> language
> >>>>>> threads and system threads.  Most important are issues of
> >>>>>> preserving
> >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
> >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
> >>>>>
> >>>>> I know that paper, and being Modula-3 camp, I am - by definition -
> >>>>> agreed to no-library-approach-for-threads :).
> >>>>>
> >>>>>>  There are
> >>>>>> also questions of portability and debuggability.
> >>>>>
> >>>>>   Of course. That's why I am using only POSIX defined features,  
> >>>>> and
> >>>>> when
> >>>>> in doubt - ones used by Boehm in his famous GC :).
> >>>>>
> >>>>>> I agree that global
> >>>>>> locks are to be avoided where they cause known contention, but
> >>>>>> there
> >>>>>> are tradeoffs there too.
> >>>>>
> >>>>>   Global lock is bad, whatever reasons.
> >>>>>
> >>>>>> For large numbers of threads (as you appear
> >>>>>> to need) I think we would need to adopt some other implementation
> >>>>>> approach, possibly by multiplexing multiple lightweight user- 
> >>>>>> level
> >>>>>> threads on some smaller number of heavyweight system-level  
> >>>>>> threads,
> >>>>>> but then you run into scheduling and load-balancing problems.
> >>>>>
> >>>>>   I've argued this before... With O(1) process scheduling  
> >>>>> available
> >>>>> for
> >>>>> four years from Linux, and in that time surely from everyone
> >>>>> else... It
> >>>>> would be bigger problem to maintain scheduling for special "BIG"
> >>>>> cases
> >>>>> inside our support libraries than to rely on operating system.  
> >>>>> It's
> >>>>> good
> >>>>> that mainstream OS people recognized threads as Need, and it is
> >>>>> time now
> >>>>> for us to accept they did it well - AT LAST.
> >>>>>
> >>>>>   And, very important... I can't see what is heavyweight on system
> >>>>> which
> >>>>> does 10,000 context switches per second for 1500 threads with  
> >>>>> 2% CPU
> >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
> >>>>> WERE
> >>>>> heavyweight four years ago, and they are not, long since. Even
> >>>>> Windows
> >>>>> has lightweight kernel-space threads :).
> >>>>>
> >>>>>>
> >>>>>>> We talked about this at least once before, and I think I  
> >>>>>>> remember
> >>>>>>> you
> >>>>>>> insisted on more compatibility than can be read from SPwM3.  
> >>>>>>> Do you
> >>>>>>> think
> >>>>>>> best idea would be to integrate mine NPTL code into CM3 for
> >>>>>>> people to
> >>>>>>> trash and test, and let everyone select what is best for their
> >>>>>>> situation?
> >>>>>>
> >>>>>> What I wanted was a situation where programs would be able to run
> >>>>>> with the same tools (e.g., showheap, showthread) under both user-
> >>>>>> level and system threading.  This goal has been achieved with the
> >>>>>> current pthread-based implementation.
> >>>>>
> >>>>>   It is good reasoning, and it's one of reasons I did not suggest
> >>>>> replacement... I think mine version is less bloated and I know  
> >>>>> it's
> >>>>> very
> >>>>> good for long and stable process uptime we all expect from  
> >>>>> Modula-3
> >>>>> programs. But also, implied compatibilities outside of SPwM3 and
> >>>>> direct
> >>>>> demands from other parts of runtime were not on my list. These  
> >>>>> I've
> >>>>> respected, and it looks like these are good production time
> >>>>> criteria. As
> >>>>> opposed to excellent development time criteria you based yours on.
> >>>>>
> >>>>>> Moreover, I wanted something
> >>>>>> where variations in thread support from one system to another  
> >>>>>> could
> >>>>>> be exploited most easily (such as for systems where thread  
> >>>>>> suspend/
> >>>>>> resume is provided as a primitive).  Again, the current
> >>>>>> implementation achieves this, and runs with minimal target- 
> >>>>>> specific
> >>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
> >>>>>> should be
> >>>>>> relatively straightforward.
> >>>>>
> >>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
> >>>>> extremmely
> >>>>> POSIX, I see no problem.
> >>>>>
> >>>>>>
> >>>>>>> Problems I had with my pthread implementation were all related
> >>>>>>> to VM
> >>>>>>> hell of earlier GC implementation... After you did that piece
> >>>>>>> of art
> >>>>>>> with new approach to GC, I expect infinite uptimes from my
> >>>>>>> servers and
> >>>>>>> bots :). Big thanks for that!
> >>>>>>
> >>>>>> Any native threading implementation is going to have problems  
> >>>>>> with
> >>>>>> VM-
> >>>>>> based memory management.  I'm surprised to were able to get
> >>>>>> anything
> >>>>>> going with the VM-based GC.
> >>>>>
> >>>>>   Anything is pretty much - I have heavy multithreaded servers
> >>>>> running
> >>>>> literally for years,,, one of them is up since January of 2004, it
> >>>>> services few hundreds of connected users (and up to 1500  
> >>>>> threads) at
> >>>>> almost every moment and breaks only when system reboots :). All  
> >>>>> that
> >>>>> with heavy integration of various C libraries.
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> dd
> >>>>>>> -- 
> >>>>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>>>
> >>>>> -- 
> >>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>
> >>> -- 
> >>> Dragi?a Duri? <dragisha at m3w.org>
> >>
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Tue May 22 15:13:09 2007
From: darko at darko.org (Darko)
Date: Tue, 22 May 2007 15:13:09 +0200
Subject: [M3devel] Darwin Intel Data Alignment
Message-ID: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>

Hi,

I'm having problems trying to use byte aligned fields in structures  
in the new Darwin Intel compiler. My understanding is that the  
processor accepts accesses on (word, etc) data not aligned on their  
normal boundaries. Is there something in the back end which means  
this is not possible?

The <*LAZYALIGN*> pragma doesn't work properly and I get:

"../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
unaligned store_indirect  type=11  s/a=32/16
"../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
to find integer type?  type=Int.32  s/o/a=32/64/16

when I compile. I think the platform independent alignment  
calculation code is correct, so I'm thinking this is a back end  
limitation.

Any thoughts?


Cheers,
Darko.



From hosking at cs.purdue.edu  Tue May 22 17:22:55 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 22 May 2007 11:22:55 -0400
Subject: [M3devel] Darwin Intel Data Alignment
In-Reply-To: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
References: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
Message-ID: <ACCD1401-62C4-4E4A-AD00-CFF550072A91@cs.purdue.edu>

Hmm.  I am not familiar with this aspect of things.  One thing would  
be to check if things work for you on LINUXLIBC6.  Then we just need  
to figure out what is different between the Darwin and Linux Intel  
targets.  Can you send me some test code that demonstrates the problem?

On May 22, 2007, at 9:13 AM, Darko wrote:

> Hi,
>
> I'm having problems trying to use byte aligned fields in structures  
> in the new Darwin Intel compiler. My understanding is that the  
> processor accepts accesses on (word, etc) data not aligned on their  
> normal boundaries. Is there something in the back end which means  
> this is not possible?
>
> The <*LAZYALIGN*> pragma doesn't work properly and I get:
>
> "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
> unaligned store_indirect  type=11  s/a=32/16
> "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
> to find integer type?  type=Int.32  s/o/a=32/64/16
>
> when I compile. I think the platform independent alignment  
> calculation code is correct, so I'm thinking this is a back end  
> limitation.
>
> Any thoughts?
>
>
> Cheers,
> Darko.



From rodney.bates at wichita.edu  Sat May 26 23:48:17 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sat, 26 May 2007 16:48:17 -0500
Subject: [M3devel] Still can't bootstrap CM3
Message-ID: <4658AB21.5030100@wichita.edu>

I still can't get a working cm3 built. I tried the following:

Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz
and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there.

Did 'cvs -z3 update -dAP' in the top of my copy of cm3.

In scripts, did 'do-cm3-min.sh buildship', which ended with:

-----------------------------------------------------------------------------
=== package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  && cm3 -ship 
-DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

new source -> compiling ConvertPacking.m3
new "ConvertPacking.mo" -> archiving libm3.a

Fatal Error: incomplete library

ThreadPosix.i3: missing object types:  _t1541f475
imported by:  TimeStamp.m3
  *** execution of  failed ***

------------------------------------------------------------------------------

I believe this is the same symptom I had the last time I tried.

-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From rodney.bates at wichita.edu  Sun May 27 18:23:51 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 27 May 2007 11:23:51 -0500
Subject: [M3devel] Can't load dynamically linked m3bundle
Message-ID: <4659B097.5040908@wichita.edu>

While trying to bootstrap CM3, I get this:

-----------------------------------------------------------------------------------------------------
rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle
-rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle
rodney at runnymede:/usr/local/cm3/bin$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically 
linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle
-bash: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle
/usr/bin/ldd: line 125: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue
Mandriva Linux release 2007.0 (Cooker) for i586
Kernel 2.6.17-5mdv on an i686 / \l
rodney at runnymede:/usr/local/cm3/bin$
------------------------------------------------------------------------------------------------------

On an earlier Mandriva release, m3bundle turns out statically linked.  In both cases,
the m3makefile for m3bundle contains 'build_standalone ( )'.

------------------------------------------------------------------------------------------------------
[rodney at selkirk bin]$ pwd
/usr/local/cm3/bin
[rodney at selkirk bin]$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically 
linked, not stripped
[rodney at selkirk bin]$ cat /etc/issue
Mandrakelinux release 10.2 (Limited Edition 2005) for i586
Kernel 2.6.11-6mdk on an i686 / \l
[rodney at selkirk bin]$
------------------------------------------------------------------------------------------------------

I am guessing that there is some dynamic library that m3bundle wants that I don't have
installed, but how can I find it?  'objdump -T m3bundle' gives a lot of entries, the
most interesting ones naming GCC_3.0.  The system where it fails has gcc 4.1.1 installed.
I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a
symbolic link to it, but to no avail.


-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From hosking at cs.purdue.edu  Mon May 28 18:55:21 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 28 May 2007 12:55:21 -0400
Subject: [M3devel] Still can't bootstrap CM3
In-Reply-To: <4658AB21.5030100@wichita.edu>
References: <4658AB21.5030100@wichita.edu>
Message-ID: <D880263B-B69A-4A92-BE10-1E5A3F940355@cs.purdue.edu>

Did you do realclean before buildship?

I have successfully bootstrapped from a clean CVS head using that cm3  
binary on several Linux platforms.

On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote:

> I still can't get a working cm3 built. I tried the following:
>
> Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ 
> pub/hosking/m3/LINUXLIBC6/cm3.gz
> and put it in /usr/local/cm3/bin, which is on my path and the  
> command 'cm3' finds it there.
>
> Did 'cvs -z3 update -dAP' in the top of my copy of cm3.
>
> In scripts, did 'do-cm3-min.sh buildship', which ended with:
>
> ---------------------------------------------------------------------- 
> -------
> === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
>  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  &&  
> cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> new source -> compiling ConvertPacking.m3
> new "ConvertPacking.mo" -> archiving libm3.a
>
> Fatal Error: incomplete library
>
> ThreadPosix.i3: missing object types:  _t1541f475
> imported by:  TimeStamp.m3
>  *** execution of  failed ***
>
> ---------------------------------------------------------------------- 
> --------
>
> I believe this is the same symptom I had the last time I tried.
>
> -- 
> -------------------------------------------------------------
> Rodney M. Bates, retired assistant professor
> Dept. of Computer Science, Wichita State University
> Wichita, KS 67260-0083
> 316-978-3922
> rodney.bates at wichita.edu



From ronny.forberger at elegosoft.com  Wed May 30 15:24:16 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 30 May 2007 15:24:16 +0200
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
Message-ID: <465D7B00.8000205@elegosoft.com>

Dear m3-Developers,

I could not successfully build the cm3 compiler when invoking

   

> ./do-cm3-std.sh buildship


from the script directory as described in the documentation.

The build process stops by the following:

> === package /root/cm3/m3-sys/m3cgcat ===
>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
>  -> linking m3cgcat
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InnerLockMutex':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined 
> reference to `pthread_cond_wait'
>
[ some similar lines removed ]

>
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PushEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined 
> reference to `pthread_getspecific'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PopEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InitHandlers':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined 
> reference to `pthread_key_create'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined 
> reference to `pthread_setspecific'
> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In 
> function `vfork':
> (.text+0x1): undefined reference to `pthread_create'
> collect2: ld returned 1 exit status
> Fatal Error: package build failed
>  *** execution of  failed ***


Looks like gcc cannot link against the pthread-lib.

I have already added

> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]

to the SYSTEM_LIBS array in cm3.cfg.

This is a Debian 4.0  'etch' system (x86) where libpthread.a resides in 
/usr/lib.

I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread 
arccordingly which become linked against libpthread.a properly.


Do you have any glues?

Best regards,

Ronny

-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From hosking at cs.purdue.edu  Wed May 30 16:47:56 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 30 May 2007 10:47:56 -0400
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
References: <465D7B00.8000205@elegosoft.com>
Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu>

Can you use the -commands option to cm3 to show the link command?  It  
certainly looks like it is not finding the pthread library.

On May 30, 2007, at 9:24 AM, Ronny Forberger wrote:

> Dear m3-Developers,
>
> I could not successfully build the cm3 compiler when invoking
>
>
>> ./do-cm3-std.sh buildship
>
>
> from the script directory as described in the documentation.
>
> The build process stops by the following:
>
>> === package /root/cm3/m3-sys/m3cgcat ===
>>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/ 
>> cm3'  +++
>> --- building in LINUXLIBC6 ---
>>
>> ignoring ../src/m3overrides
>>
>>  -> linking m3cgcat
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:  
>> undefined reference to `pthread_cond_wait'
>>
> [ some similar lines removed ]
>
>>
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PushEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:  
>> undefined reference to `pthread_getspecific'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PopEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:  
>> undefined reference to `pthread_key_create'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:  
>> undefined reference to `pthread_setspecific'
>> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):  
>> In function `vfork':
>> (.text+0x1): undefined reference to `pthread_create'
>> collect2: ld returned 1 exit status
>> Fatal Error: package build failed
>>  *** execution of  failed ***
>
>
> Looks like gcc cannot link against the pthread-lib.
>
> I have already added
>
>> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
>
> to the SYSTEM_LIBS array in cm3.cfg.
>
> This is a Debian 4.0  'etch' system (x86) where libpthread.a  
> resides in /usr/lib.
>
> I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread  
> arccordingly which become linked against libpthread.a properly.
>
>
> Do you have any glues?
>
> Best regards,
>
> Ronny
>
> -- 
> Ronny Forberger
> Systemadministrator
>
> elego Software Solutions GmbH           ronny.forberger(at) 
> elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
> http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner
>
> Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) 
> elego.de .



From dabenavidesd at yahoo.es  Thu May 31 01:42:04 2007
From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.)
Date: Thu, 31 May 2007 01:42:04 +0200 (CEST)
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com>

Hi,
Looks that you haven't used the SYSTEM_LIBORDER array,
must be "PTHREAD" there in order to really link it.

Maybe this little guide can help. Can have several
omissions, but It works well in (K)ubuntu 6.06 and
6.10 
http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000


--- Ronny Forberger <ronny.forberger at elegosoft.com>
wrote:
> Dear m3-Developers,
> 
> I could not successfully build the cm3 compiler when
> invoking
> 
>    
> 
> > ./do-cm3-std.sh buildship
> 
> 
> from the script directory as described in the
> documentation.
> 
> The build process stops by the following:
> 
> > === package /root/cm3/m3-sys/m3cgcat ===
> >  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship
> -DROOT='/root/cm3'  +++
> > --- building in LINUXLIBC6 ---
> >
> > ignoring ../src/m3overrides
> >
> >  -> linking m3cgcat
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InnerLockMutex':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:
> undefined 
> > reference to `pthread_cond_wait'
> >
> [ some similar lines removed ]
> 
> >
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PushEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:
> undefined 
> > reference to `pthread_getspecific'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PopEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InitHandlers':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:
> undefined 
> > reference to `pthread_key_create'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):
> In 
> > function `vfork':
> > (.text+0x1): undefined reference to
> `pthread_create'
> > collect2: ld returned 1 exit status
> > Fatal Error: package build failed
> >  *** execution of  failed ***
> 
> 
> Looks like gcc cannot link against the pthread-lib.
> 
> I have already added
> 
> > "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
> 
> to the SYSTEM_LIBS array in cm3.cfg.
> 
> This is a Debian 4.0  'etch' system (x86) where
> libpthread.a resides in 
> /usr/lib.
> 
> I can build trivial C programs like gcc foo.c -L
> /usr/lib -lpthread 
> arccordingly which become linked against
> libpthread.a properly.
> 
> 
> Do you have any glues?
> 
> Best regards,
> 
> Ronny
> 
> -- 
> Ronny Forberger
> Systemadministrator
> 
> elego Software Solutions GmbH          
> ronny.forberger(at)elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht
> Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der
> Gesellschaft: Berlin
> http://www.elegosoft.com               
> Gesch?ftsf?hrer: Olaf Wagner
> 
> Bitte senden Sie Ihre technischen Support-Anfragen
> an admins(at)elego.de .
> 
> 



		
______________________________________________ 
LLama Gratis a cualquier PC del Mundo. 
Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. 
http://es.voice.yahoo.com


From dragisha at m3w.org  Wed May  2 07:45:52 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 07:45:52 +0200
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
Message-ID: <1178084752.8631.3.camel@faramir.m3w.org>

  Is it possible to get this fix separatelly from threading fixes? It's
more important, esp since only program that crashes threading is
TestThreads :)

  TIA,
  dd

On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
> Yes, this fix is in CVS head.
> 
> 
> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
> 
> > It looks like this fix never came through... With cvs-head CM3,
> > unpickling of TextRefTbl.T instance still does not work.
> >
> > dd
> >
> > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
> >> Rodney,
> >>
> >> I've checked in fixes to RTTypeMap and clients that should restore
> >> functional pickling support.  For some reason my commit messages are
> >> awaiting moderator approval (it seems my commit id is not the same as
> >> my subscription to the m3commit list) before they can be sent out.
> >> Please make sure that unpickling now works for you.
> >>
> >> Best regards,
> >>
> >> Tony
> >>
> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
> >>
> >>> I tracked down a bug unpickling to the following.  When executing
> >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
> >>>
> >>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
> >>>                   ref^ := v.readRef(nelem.refType);
> >>>                 END;
> >>>
> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref).
> >>> CheckStoreTraced expects ref to be a normal reference to an object,
> >>> computes the address of its header (4 bytes less), and sets the
> >>> dirty bit.
> >>>
> >>> But ConvertPacking is reading in to a field of a recently created
> >>> object that is not first in the object.  It computes ref to point
> >>> to the field, and CheckStoreTraced actually steps on the previous
> >>> real field instead of the header.
> >>>
> >>> It looks like the actions of CheckStoreTraced are needed, but I
> >>> don't right off hand see how to recode Convert to fix this.  It's
> >>> use of ref can't be moved.  Presumably, there could be other unsafe
> >>> code elsewhere using the same technique to store data.
> >>>
> >>> Does the compiler need a different criterion on when to generate
> >>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
> >>> still is pointing inside a traced object.   Maybe people who modify
> >>> objects in this way need to explicitly call CheckStoreTraced?
> >>>
> >>> -- 
> >>> -------------------------------------------------------------
> >>> Rodney M. Bates, retired assistant professor
> >>> Dept. of Computer Science, Wichita State University
> >>> Wichita, KS 67260-0083
> >>> 316-978-3922
> >>> rodney.bates at wichita.edu
> >>> _______________________________________________
> >>> M3devel mailing list
> >>> M3devel at elegosoft.com
> >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> >>
> >> Antony Hosking                | Associate Professor
> >> Dept of Computer Science      | Office: +1 765 494-6001
> >> Purdue University             | Mobile: +1 765 427-5484
> >> 250 N. University Street      | Email:  hosking at cs.purdue.edu
> >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
> >>   _--_|\
> >> /      \
> >> \_.--._/    )
> >>        v    /
> >>
> >>
> >>
> >> _______________________________________________
> >> M3devel mailing list
> >> M3devel at elegosoft.com
> >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May  2 08:18:22 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 2 May 2007 02:18:22 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
	<1178084752.8631.3.camel@faramir.m3w.org>
Message-ID: <F3AF7610-4655-4662-A530-ECCE833A1464@cs.purdue.edu>

Just grab CVS from before 2007/03/2007 (when I broke GC).  The  
pertinent fix was to RTTypeMap.m3, as at the current CVS head, which  
is dated 2007/01/09.

Thread/GC-related fixes are coming -- I know what the problem is but  
am in the middle of refactoring the heap allocation code.

On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote:

>   Is it possible to get this fix separatelly from threading fixes?  
> It's
> more important, esp since only program that crashes threading is
> TestThreads :)
>
>   TIA,
>   dd
>
> On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
>> Yes, this fix is in CVS head.
>>
>>
>> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
>>
>>> It looks like this fix never came through... With cvs-head CM3,
>>> unpickling of TextRefTbl.T instance still does not work.
>>>
>>> dd
>>>
>>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
>>>> Rodney,
>>>>
>>>> I've checked in fixes to RTTypeMap and clients that should restore
>>>> functional pickling support.  For some reason my commit messages  
>>>> are
>>>> awaiting moderator approval (it seems my commit id is not the  
>>>> same as
>>>> my subscription to the m3commit list) before they can be sent out.
>>>> Please make sure that unpickling now works for you.
>>>>
>>>> Best regards,
>>>>
>>>> Tony
>>>>
>>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
>>>>
>>>>> I tracked down a bug unpickling to the following.  When executing
>>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
>>>>>
>>>>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
>>>>>                   ref^ := v.readRef(nelem.refType);
>>>>>                 END;
>>>>>
>>>>> has a compiler-generated call on RTCollector.CheckStoreTraced 
>>>>> (ref).
>>>>> CheckStoreTraced expects ref to be a normal reference to an  
>>>>> object,
>>>>> computes the address of its header (4 bytes less), and sets the
>>>>> dirty bit.
>>>>>
>>>>> But ConvertPacking is reading in to a field of a recently created
>>>>> object that is not first in the object.  It computes ref to point
>>>>> to the field, and CheckStoreTraced actually steps on the previous
>>>>> real field instead of the header.
>>>>>
>>>>> It looks like the actions of CheckStoreTraced are needed, but I
>>>>> don't right off hand see how to recode Convert to fix this.  It's
>>>>> use of ref can't be moved.  Presumably, there could be other  
>>>>> unsafe
>>>>> code elsewhere using the same technique to store data.
>>>>>
>>>>> Does the compiler need a different criterion on when to generate
>>>>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
>>>>> still is pointing inside a traced object.   Maybe people who  
>>>>> modify
>>>>> objects in this way need to explicitly call CheckStoreTraced?
>>>>>
>>>>> -- 
>>>>> -------------------------------------------------------------
>>>>> Rodney M. Bates, retired assistant professor
>>>>> Dept. of Computer Science, Wichita State University
>>>>> Wichita, KS 67260-0083
>>>>> 316-978-3922
>>>>> rodney.bates at wichita.edu
>>>>> _______________________________________________
>>>>> M3devel mailing list
>>>>> M3devel at elegosoft.com
>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>> Antony Hosking                | Associate Professor
>>>> Dept of Computer Science      | Office: +1 765 494-6001
>>>> Purdue University             | Mobile: +1 765 427-5484
>>>> 250 N. University Street      | Email:  hosking at cs.purdue.edu
>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
>>>>   _--_|\
>>>> /      \
>>>> \_.--._/    )
>>>>        v    /
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From ronny.forberger at elegosoft.com  Wed May  2 11:24:14 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 02 May 2007 11:24:14 +0200
Subject: [M3devel] Just a test - please ignore
Message-ID: <463858BE.1040000@elegosoft.com>


-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From dragisha at m3w.org  Wed May  2 23:11:42 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 23:11:42 +0200
Subject: [M3devel] formals initialization... NIL?
Message-ID: <1178140302.8631.26.camel@faramir.m3w.org>

99      PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args:
ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User }
=
100       BEGIN
101         IF args[FIRST(args)] = NIL THEN
102           RAISE User(
103             NEW(RefSeq.T).fromArray(
104               ARRAY OF REFANY { e.getRef(f), e.getQName()}
105             )
(m3gdb) p args
$1 = [16_00000001]

Is this normal? 16_00000001 is NIL? Why not printed as such?

-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May  5 09:44:07 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 05 May 2007 09:44:07 +0200
Subject: [M3devel] status of -linkall?
Message-ID: <1178351048.21091.3.camel@faramir.m3w.org>

I have a situation where -linkall works "as advertised", at least I
don't see anything missing yet :). Same set of modules, when included in
build of another program, fails to initialize...

If I've let anyone forget it :), it's plugin architecture where modules
plug to appropriate "connection points" simply by inclusion in
m3makefile.

Is it known problem with -linkall?

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Mon May  7 08:03:45 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 07 May 2007 08:03:45 +0200
Subject: [M3devel] import/exports loops... dynamic loading of Modula-3
	modules
Message-ID: <1178517825.3925.23.camel@faramir.m3w.org>

I am thinking about writing some loop detecting tool for (in)famous
order-of-initialization sindrome. Anyone did it already? I think we
really need something to help sort out of EXPORTS nightmare and it's
influence on RTLinker behaviour.

Also in news :), after I've tried to solve my problems with -linkall
(and got very big executabales as consequence, together with almost
working programs - at least until I try to make another similar build :)
- dynamic loading of modules was logical next step.

I've done it, and can confirm RTLinker.AddUnit works as advertised, any
time one calls it :) It also solves my plugin architecture problem, as
my plugin modules are not imported directly and thus their
initialization is fully performed with adequate RTLinker.AddUnit call.

If there's interest in how-to-make-dynamic-modules-happen, I can post it
somewhere. My google search did not show ay Modula-3 work on that.

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Wed May  9 01:55:18 2007
From: darko at darko.org (Darko)
Date: Wed, 9 May 2007 01:55:18 +0200
Subject: [M3devel] m3.org expiry
Message-ID: <B588AC8B-D2DD-4271-B67D-4CC95C8E6F02@darko.org>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.



From wagner at elegosoft.com  Wed May  9 07:57:41 2007
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 9 May 2007 07:57:41 +0200
Subject: [M3devel] [darko@darko.org:  m3.org expiry]
Message-ID: <20070509055740.GA20162@elegosoft.com>

Hi Farshad,

I seem to remember that you tried to register m3.org for a period
longer than a year, but now it's gone again:

% host www.m3.org
www.m3.org is an alias for resalehost.networksolutions.com.
resalehost.networksolutions.com has address 205.178.189.128

What's the current state of things from your side? Should anybody
else get and host m3.org?

Olaf

----- Forwarded message from Darko <darko at darko.org> -----

X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
To: Modula-3 developers <m3devel at elegosoft.com>
From: Darko <darko at darko.org>
Date: Wed, 9 May 2007 01:55:18 +0200
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed 
	version=3.0.3
X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de
X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de
X-Virus-Status: Clean
Subject: [M3devel] m3.org expiry
X-BeenThere: m3devel at elegosoft.com
X-Mailman-Version: 2.1.9
List-Id: <m3devel.elegosoft.com>
List-Unsubscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=unsubscribe>
List-Archive: <http://mail.elegosoft.com/pipermail/m3devel>
List-Post: <mailto:m3devel at elegosoft.com>
List-Help: <mailto:m3devel-request at elegosoft.com?subject=help>
List-Subscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=subscribe>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel

----- End forwarded message -----

-- 
Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 23 45 869  fax: +49 30 23 45 86 95
http://www.elegosoft.com  |  Gesch?ftsf?hrer: Olaf Wagner  |   Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194


From hosking at cs.purdue.edu  Wed May 16 17:21:12 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:21:12 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <A79828E1-0986-4552-B0A1-C93811547613@cs.purdue.edu>

TestThreads is still locking up with 3000 threads (it works with 2000  
on my I386_DARWIN box).  I am now tracking this down.

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May 16 17:19:54 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:19:54 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org>
References: <1177277134.737.47.camel@faramir.m3w.org>
	<9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu>
	<19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu>
	<1177281325.737.55.camel@faramir.m3w.org>
	<F32E2364-A890-4739-AC5F-D1E46616C503@cs.purdue.edu>
	<1177308751.737.63.camel@faramir.m3w.org>
Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu>

On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote:

> Thank you. I am using your posted compiler for now.
>
> dd
>
> On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote:
>> Something appears to be very broken with my most recent fixes for
>> your threading problems (since Apr 19, when I put the last cm3.gz
>> compiler bootstrap up).  I will try to track this down tomorrow.

I have tracked down and fixed this brokenness.  CVS head should now  
behave reasonably.

>>
>> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote:
>>
>>> Any pointer on how to do this?
>>>
>>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am
>>> using.
>>>
>>> dd
>>>
>>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote:
>>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt
>>>> for this fix to take effect.
>>>>
>>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote:
>>>>
>>>>> This works just fine for me on the most recent CVS head for
>>>>> I386_DARWIN.
>>>>>
>>>>> I wonder if you have not updated your compiler to match?  There  
>>>>> was
>>>>> a matching fix to the compiler.  You should build and ship:
>>>>>
>>>>> m3gc-simple
>>>>> m3core
>>>>> libm3
>>>>> m3middle
>>>>> m3linker
>>>>> m3front
>>>>> m3quake
>>>>> cm3
>>>>>
>>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your
>>>>> cm3.cfg).  This will get you a new compiler.
>>>>>
>>>>> Then, rebuild your CM3 installation from the CVS head.
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: Dragi?a Duri? <dragisha at m3w.org>
>>>>>> Date: April 22, 2007 5:25:33 PM EDT
>>>>>> To: Tony Hosking <hosking at cs.purdue.edu>
>>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug
>>>>>>
>>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote:
>>>>>>> Can you send me a stack dump from gdb at the point of the error?
>>>>>>> Even better, do you have a simple example program I can use to
>>>>>>> test
>>>>>>> with?
>>>>>>
>>>>>> Here is one. Quick and dirty, but it shows problem.
>>>>>>
>>>>>> -- 
>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>> <Pickles.m3>
>>>>>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Fri May 18 13:50:24 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Fri, 18 May 2007 13:50:24 +0200
Subject: [M3devel] Pickle compatibility...
Message-ID: <1179489025.6698.3.camel@faramir.m3w.org>

I have some data pickled by pm3 pickler... Now, reading these fails...
First app I've tried had online quiz high scores pickled, but now I have
to convert billing app :). What can I do to convert this?

dd

-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Sat May 19 22:00:36 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 19 May 2007 16:00:36 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>

I just checked in a patch that is closer to your code, and that lets  
me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
creation fails for that many).

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May 19 22:16:47 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 19 May 2007 22:16:47 +0200
Subject: [M3devel] pthread in cm3...
In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
	<0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
Message-ID: <1179605808.8468.4.camel@faramir.m3w.org>

I am running 4000, but it's not most important anyway...

Did you see that (Un)LockMutex semantics I've enabled so one can
overload methods invoked by LOCK .. DO?

dd

On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote:
> I just checked in a patch that is closer to your code, and that lets  
> me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
> creation fails for that many).
> 
> On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:
> 
> > After implementing that workaround for result code 11 in that  
> > SIGSUSPEND
> > loop, every time during 1st or at most second pass (of 4000) it  
> > stucks.
> > Not same place every time, though... I think there are RCs in
> > LockHeap/SuspendAll logic.
> >
> > dd
> >
> > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
> >> Yes, good thinking.  Tuning the threads systems is a good plan all
> >> round.
> >>
> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
> >>
> >>> Just a random thought...
> >>>
> >>> I don't think my TestThreads is something special, but it's few  
> >>> thread
> >>> use patterns combined... And I've just had bright :) idea  
> >>> yesterday -
> >>> it's also decent benchmark for whole threading system... I think it
> >>> would be nice to test it with 10000 rounds, 4000 threads each (once
> >>> cm3
> >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
> >>> tests for
> >>> mine.
> >>>
> >>> I think these extra data structures and global locks in PTHREAD are
> >>> big
> >>> efficiency killers. Benchmark will show.
> >>>
> >>> dd
> >>>
> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
> >>>> I take all of your points seriously.  One option would be to offer
> >>>> your threads implementation as another build option for CM3.  I'm
> >>>> going to track down the bug I introduced recently and then we can
> >>>> consider how to move forward.
> >>>>
> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
> >>>>
> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
> >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
> >>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> Have been skimming through source of PTHREAD code... And I think
> >>>>>>> job can
> >>>>>>> be done without so much relying on how-they-did-it-before, esp
> >>>>>>> with
> >>>>>>> regard to list of waiters and similar internal and global
> >>>>>>> structures.
> >>>>>>> Also, I see number of global locks and I am sure they are
> >>>>>>> congestion
> >>>>>>> generators every now and while, esp in heavy threading  
> >>>>>>> situations.
> >>>>>>>
> >>>>>>> Of course, there is number of approaches to this multi-thread
> >>>>>>> situations. Mine being one of very nonconservative use of
> >>>>>>> threads, I
> >>>>>>> think it is important to remain open to possibly very big
> >>>>>>> number of
> >>>>>>> threads running in single process - meaning scalability is  
> >>>>>>> one of
> >>>>>>> primary objections... As global locks don't do well with
> >>>>>>> scalability, I
> >>>>>>> don't like "cm" and similar global congestion points.
> >>>>>>
> >>>>>> Yes, there are tensions between a thin/absent veneer between
> >>>>>> language
> >>>>>> threads and system threads.  Most important are issues of
> >>>>>> preserving
> >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
> >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
> >>>>>
> >>>>> I know that paper, and being Modula-3 camp, I am - by definition -
> >>>>> agreed to no-library-approach-for-threads :).
> >>>>>
> >>>>>>  There are
> >>>>>> also questions of portability and debuggability.
> >>>>>
> >>>>>   Of course. That's why I am using only POSIX defined features,  
> >>>>> and
> >>>>> when
> >>>>> in doubt - ones used by Boehm in his famous GC :).
> >>>>>
> >>>>>> I agree that global
> >>>>>> locks are to be avoided where they cause known contention, but
> >>>>>> there
> >>>>>> are tradeoffs there too.
> >>>>>
> >>>>>   Global lock is bad, whatever reasons.
> >>>>>
> >>>>>> For large numbers of threads (as you appear
> >>>>>> to need) I think we would need to adopt some other implementation
> >>>>>> approach, possibly by multiplexing multiple lightweight user- 
> >>>>>> level
> >>>>>> threads on some smaller number of heavyweight system-level  
> >>>>>> threads,
> >>>>>> but then you run into scheduling and load-balancing problems.
> >>>>>
> >>>>>   I've argued this before... With O(1) process scheduling  
> >>>>> available
> >>>>> for
> >>>>> four years from Linux, and in that time surely from everyone
> >>>>> else... It
> >>>>> would be bigger problem to maintain scheduling for special "BIG"
> >>>>> cases
> >>>>> inside our support libraries than to rely on operating system.  
> >>>>> It's
> >>>>> good
> >>>>> that mainstream OS people recognized threads as Need, and it is
> >>>>> time now
> >>>>> for us to accept they did it well - AT LAST.
> >>>>>
> >>>>>   And, very important... I can't see what is heavyweight on system
> >>>>> which
> >>>>> does 10,000 context switches per second for 1500 threads with  
> >>>>> 2% CPU
> >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
> >>>>> WERE
> >>>>> heavyweight four years ago, and they are not, long since. Even
> >>>>> Windows
> >>>>> has lightweight kernel-space threads :).
> >>>>>
> >>>>>>
> >>>>>>> We talked about this at least once before, and I think I  
> >>>>>>> remember
> >>>>>>> you
> >>>>>>> insisted on more compatibility than can be read from SPwM3.  
> >>>>>>> Do you
> >>>>>>> think
> >>>>>>> best idea would be to integrate mine NPTL code into CM3 for
> >>>>>>> people to
> >>>>>>> trash and test, and let everyone select what is best for their
> >>>>>>> situation?
> >>>>>>
> >>>>>> What I wanted was a situation where programs would be able to run
> >>>>>> with the same tools (e.g., showheap, showthread) under both user-
> >>>>>> level and system threading.  This goal has been achieved with the
> >>>>>> current pthread-based implementation.
> >>>>>
> >>>>>   It is good reasoning, and it's one of reasons I did not suggest
> >>>>> replacement... I think mine version is less bloated and I know  
> >>>>> it's
> >>>>> very
> >>>>> good for long and stable process uptime we all expect from  
> >>>>> Modula-3
> >>>>> programs. But also, implied compatibilities outside of SPwM3 and
> >>>>> direct
> >>>>> demands from other parts of runtime were not on my list. These  
> >>>>> I've
> >>>>> respected, and it looks like these are good production time
> >>>>> criteria. As
> >>>>> opposed to excellent development time criteria you based yours on.
> >>>>>
> >>>>>> Moreover, I wanted something
> >>>>>> where variations in thread support from one system to another  
> >>>>>> could
> >>>>>> be exploited most easily (such as for systems where thread  
> >>>>>> suspend/
> >>>>>> resume is provided as a primitive).  Again, the current
> >>>>>> implementation achieves this, and runs with minimal target- 
> >>>>>> specific
> >>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
> >>>>>> should be
> >>>>>> relatively straightforward.
> >>>>>
> >>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
> >>>>> extremmely
> >>>>> POSIX, I see no problem.
> >>>>>
> >>>>>>
> >>>>>>> Problems I had with my pthread implementation were all related
> >>>>>>> to VM
> >>>>>>> hell of earlier GC implementation... After you did that piece
> >>>>>>> of art
> >>>>>>> with new approach to GC, I expect infinite uptimes from my
> >>>>>>> servers and
> >>>>>>> bots :). Big thanks for that!
> >>>>>>
> >>>>>> Any native threading implementation is going to have problems  
> >>>>>> with
> >>>>>> VM-
> >>>>>> based memory management.  I'm surprised to were able to get
> >>>>>> anything
> >>>>>> going with the VM-based GC.
> >>>>>
> >>>>>   Anything is pretty much - I have heavy multithreaded servers
> >>>>> running
> >>>>> literally for years,,, one of them is up since January of 2004, it
> >>>>> services few hundreds of connected users (and up to 1500  
> >>>>> threads) at
> >>>>> almost every moment and breaks only when system reboots :). All  
> >>>>> that
> >>>>> with heavy integration of various C libraries.
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> dd
> >>>>>>> -- 
> >>>>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>>>
> >>>>> -- 
> >>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>
> >>> -- 
> >>> Dragi?a Duri? <dragisha at m3w.org>
> >>
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Tue May 22 15:13:09 2007
From: darko at darko.org (Darko)
Date: Tue, 22 May 2007 15:13:09 +0200
Subject: [M3devel] Darwin Intel Data Alignment
Message-ID: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>

Hi,

I'm having problems trying to use byte aligned fields in structures  
in the new Darwin Intel compiler. My understanding is that the  
processor accepts accesses on (word, etc) data not aligned on their  
normal boundaries. Is there something in the back end which means  
this is not possible?

The <*LAZYALIGN*> pragma doesn't work properly and I get:

"../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
unaligned store_indirect  type=11  s/a=32/16
"../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
to find integer type?  type=Int.32  s/o/a=32/64/16

when I compile. I think the platform independent alignment  
calculation code is correct, so I'm thinking this is a back end  
limitation.

Any thoughts?


Cheers,
Darko.



From hosking at cs.purdue.edu  Tue May 22 17:22:55 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 22 May 2007 11:22:55 -0400
Subject: [M3devel] Darwin Intel Data Alignment
In-Reply-To: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
References: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
Message-ID: <ACCD1401-62C4-4E4A-AD00-CFF550072A91@cs.purdue.edu>

Hmm.  I am not familiar with this aspect of things.  One thing would  
be to check if things work for you on LINUXLIBC6.  Then we just need  
to figure out what is different between the Darwin and Linux Intel  
targets.  Can you send me some test code that demonstrates the problem?

On May 22, 2007, at 9:13 AM, Darko wrote:

> Hi,
>
> I'm having problems trying to use byte aligned fields in structures  
> in the new Darwin Intel compiler. My understanding is that the  
> processor accepts accesses on (word, etc) data not aligned on their  
> normal boundaries. Is there something in the back end which means  
> this is not possible?
>
> The <*LAZYALIGN*> pragma doesn't work properly and I get:
>
> "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
> unaligned store_indirect  type=11  s/a=32/16
> "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
> to find integer type?  type=Int.32  s/o/a=32/64/16
>
> when I compile. I think the platform independent alignment  
> calculation code is correct, so I'm thinking this is a back end  
> limitation.
>
> Any thoughts?
>
>
> Cheers,
> Darko.



From rodney.bates at wichita.edu  Sat May 26 23:48:17 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sat, 26 May 2007 16:48:17 -0500
Subject: [M3devel] Still can't bootstrap CM3
Message-ID: <4658AB21.5030100@wichita.edu>

I still can't get a working cm3 built. I tried the following:

Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz
and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there.

Did 'cvs -z3 update -dAP' in the top of my copy of cm3.

In scripts, did 'do-cm3-min.sh buildship', which ended with:

-----------------------------------------------------------------------------
=== package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  && cm3 -ship 
-DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

new source -> compiling ConvertPacking.m3
new "ConvertPacking.mo" -> archiving libm3.a

Fatal Error: incomplete library

ThreadPosix.i3: missing object types:  _t1541f475
imported by:  TimeStamp.m3
  *** execution of  failed ***

------------------------------------------------------------------------------

I believe this is the same symptom I had the last time I tried.

-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From rodney.bates at wichita.edu  Sun May 27 18:23:51 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 27 May 2007 11:23:51 -0500
Subject: [M3devel] Can't load dynamically linked m3bundle
Message-ID: <4659B097.5040908@wichita.edu>

While trying to bootstrap CM3, I get this:

-----------------------------------------------------------------------------------------------------
rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle
-rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle
rodney at runnymede:/usr/local/cm3/bin$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically 
linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle
-bash: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle
/usr/bin/ldd: line 125: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue
Mandriva Linux release 2007.0 (Cooker) for i586
Kernel 2.6.17-5mdv on an i686 / \l
rodney at runnymede:/usr/local/cm3/bin$
------------------------------------------------------------------------------------------------------

On an earlier Mandriva release, m3bundle turns out statically linked.  In both cases,
the m3makefile for m3bundle contains 'build_standalone ( )'.

------------------------------------------------------------------------------------------------------
[rodney at selkirk bin]$ pwd
/usr/local/cm3/bin
[rodney at selkirk bin]$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically 
linked, not stripped
[rodney at selkirk bin]$ cat /etc/issue
Mandrakelinux release 10.2 (Limited Edition 2005) for i586
Kernel 2.6.11-6mdk on an i686 / \l
[rodney at selkirk bin]$
------------------------------------------------------------------------------------------------------

I am guessing that there is some dynamic library that m3bundle wants that I don't have
installed, but how can I find it?  'objdump -T m3bundle' gives a lot of entries, the
most interesting ones naming GCC_3.0.  The system where it fails has gcc 4.1.1 installed.
I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a
symbolic link to it, but to no avail.


-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From hosking at cs.purdue.edu  Mon May 28 18:55:21 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 28 May 2007 12:55:21 -0400
Subject: [M3devel] Still can't bootstrap CM3
In-Reply-To: <4658AB21.5030100@wichita.edu>
References: <4658AB21.5030100@wichita.edu>
Message-ID: <D880263B-B69A-4A92-BE10-1E5A3F940355@cs.purdue.edu>

Did you do realclean before buildship?

I have successfully bootstrapped from a clean CVS head using that cm3  
binary on several Linux platforms.

On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote:

> I still can't get a working cm3 built. I tried the following:
>
> Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ 
> pub/hosking/m3/LINUXLIBC6/cm3.gz
> and put it in /usr/local/cm3/bin, which is on my path and the  
> command 'cm3' finds it there.
>
> Did 'cvs -z3 update -dAP' in the top of my copy of cm3.
>
> In scripts, did 'do-cm3-min.sh buildship', which ended with:
>
> ---------------------------------------------------------------------- 
> -------
> === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
>  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  &&  
> cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> new source -> compiling ConvertPacking.m3
> new "ConvertPacking.mo" -> archiving libm3.a
>
> Fatal Error: incomplete library
>
> ThreadPosix.i3: missing object types:  _t1541f475
> imported by:  TimeStamp.m3
>  *** execution of  failed ***
>
> ---------------------------------------------------------------------- 
> --------
>
> I believe this is the same symptom I had the last time I tried.
>
> -- 
> -------------------------------------------------------------
> Rodney M. Bates, retired assistant professor
> Dept. of Computer Science, Wichita State University
> Wichita, KS 67260-0083
> 316-978-3922
> rodney.bates at wichita.edu



From ronny.forberger at elegosoft.com  Wed May 30 15:24:16 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 30 May 2007 15:24:16 +0200
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
Message-ID: <465D7B00.8000205@elegosoft.com>

Dear m3-Developers,

I could not successfully build the cm3 compiler when invoking

   

> ./do-cm3-std.sh buildship


from the script directory as described in the documentation.

The build process stops by the following:

> === package /root/cm3/m3-sys/m3cgcat ===
>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
>  -> linking m3cgcat
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InnerLockMutex':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined 
> reference to `pthread_cond_wait'
>
[ some similar lines removed ]

>
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PushEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined 
> reference to `pthread_getspecific'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PopEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InitHandlers':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined 
> reference to `pthread_key_create'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined 
> reference to `pthread_setspecific'
> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In 
> function `vfork':
> (.text+0x1): undefined reference to `pthread_create'
> collect2: ld returned 1 exit status
> Fatal Error: package build failed
>  *** execution of  failed ***


Looks like gcc cannot link against the pthread-lib.

I have already added

> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]

to the SYSTEM_LIBS array in cm3.cfg.

This is a Debian 4.0  'etch' system (x86) where libpthread.a resides in 
/usr/lib.

I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread 
arccordingly which become linked against libpthread.a properly.


Do you have any glues?

Best regards,

Ronny

-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From hosking at cs.purdue.edu  Wed May 30 16:47:56 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 30 May 2007 10:47:56 -0400
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
References: <465D7B00.8000205@elegosoft.com>
Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu>

Can you use the -commands option to cm3 to show the link command?  It  
certainly looks like it is not finding the pthread library.

On May 30, 2007, at 9:24 AM, Ronny Forberger wrote:

> Dear m3-Developers,
>
> I could not successfully build the cm3 compiler when invoking
>
>
>> ./do-cm3-std.sh buildship
>
>
> from the script directory as described in the documentation.
>
> The build process stops by the following:
>
>> === package /root/cm3/m3-sys/m3cgcat ===
>>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/ 
>> cm3'  +++
>> --- building in LINUXLIBC6 ---
>>
>> ignoring ../src/m3overrides
>>
>>  -> linking m3cgcat
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:  
>> undefined reference to `pthread_cond_wait'
>>
> [ some similar lines removed ]
>
>>
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PushEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:  
>> undefined reference to `pthread_getspecific'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PopEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:  
>> undefined reference to `pthread_key_create'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:  
>> undefined reference to `pthread_setspecific'
>> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):  
>> In function `vfork':
>> (.text+0x1): undefined reference to `pthread_create'
>> collect2: ld returned 1 exit status
>> Fatal Error: package build failed
>>  *** execution of  failed ***
>
>
> Looks like gcc cannot link against the pthread-lib.
>
> I have already added
>
>> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
>
> to the SYSTEM_LIBS array in cm3.cfg.
>
> This is a Debian 4.0  'etch' system (x86) where libpthread.a  
> resides in /usr/lib.
>
> I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread  
> arccordingly which become linked against libpthread.a properly.
>
>
> Do you have any glues?
>
> Best regards,
>
> Ronny
>
> -- 
> Ronny Forberger
> Systemadministrator
>
> elego Software Solutions GmbH           ronny.forberger(at) 
> elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
> http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner
>
> Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) 
> elego.de .



From dabenavidesd at yahoo.es  Thu May 31 01:42:04 2007
From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.)
Date: Thu, 31 May 2007 01:42:04 +0200 (CEST)
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com>

Hi,
Looks that you haven't used the SYSTEM_LIBORDER array,
must be "PTHREAD" there in order to really link it.

Maybe this little guide can help. Can have several
omissions, but It works well in (K)ubuntu 6.06 and
6.10 
http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000


--- Ronny Forberger <ronny.forberger at elegosoft.com>
wrote:
> Dear m3-Developers,
> 
> I could not successfully build the cm3 compiler when
> invoking
> 
>    
> 
> > ./do-cm3-std.sh buildship
> 
> 
> from the script directory as described in the
> documentation.
> 
> The build process stops by the following:
> 
> > === package /root/cm3/m3-sys/m3cgcat ===
> >  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship
> -DROOT='/root/cm3'  +++
> > --- building in LINUXLIBC6 ---
> >
> > ignoring ../src/m3overrides
> >
> >  -> linking m3cgcat
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InnerLockMutex':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:
> undefined 
> > reference to `pthread_cond_wait'
> >
> [ some similar lines removed ]
> 
> >
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PushEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:
> undefined 
> > reference to `pthread_getspecific'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PopEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InitHandlers':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:
> undefined 
> > reference to `pthread_key_create'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):
> In 
> > function `vfork':
> > (.text+0x1): undefined reference to
> `pthread_create'
> > collect2: ld returned 1 exit status
> > Fatal Error: package build failed
> >  *** execution of  failed ***
> 
> 
> Looks like gcc cannot link against the pthread-lib.
> 
> I have already added
> 
> > "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
> 
> to the SYSTEM_LIBS array in cm3.cfg.
> 
> This is a Debian 4.0  'etch' system (x86) where
> libpthread.a resides in 
> /usr/lib.
> 
> I can build trivial C programs like gcc foo.c -L
> /usr/lib -lpthread 
> arccordingly which become linked against
> libpthread.a properly.
> 
> 
> Do you have any glues?
> 
> Best regards,
> 
> Ronny
> 
> -- 
> Ronny Forberger
> Systemadministrator
> 
> elego Software Solutions GmbH          
> ronny.forberger(at)elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht
> Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der
> Gesellschaft: Berlin
> http://www.elegosoft.com               
> Gesch?ftsf?hrer: Olaf Wagner
> 
> Bitte senden Sie Ihre technischen Support-Anfragen
> an admins(at)elego.de .
> 
> 



		
______________________________________________ 
LLama Gratis a cualquier PC del Mundo. 
Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. 
http://es.voice.yahoo.com


From dragisha at m3w.org  Wed May  2 07:45:52 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 07:45:52 +0200
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
Message-ID: <1178084752.8631.3.camel@faramir.m3w.org>

  Is it possible to get this fix separatelly from threading fixes? It's
more important, esp since only program that crashes threading is
TestThreads :)

  TIA,
  dd

On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
> Yes, this fix is in CVS head.
> 
> 
> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
> 
> > It looks like this fix never came through... With cvs-head CM3,
> > unpickling of TextRefTbl.T instance still does not work.
> >
> > dd
> >
> > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
> >> Rodney,
> >>
> >> I've checked in fixes to RTTypeMap and clients that should restore
> >> functional pickling support.  For some reason my commit messages are
> >> awaiting moderator approval (it seems my commit id is not the same as
> >> my subscription to the m3commit list) before they can be sent out.
> >> Please make sure that unpickling now works for you.
> >>
> >> Best regards,
> >>
> >> Tony
> >>
> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
> >>
> >>> I tracked down a bug unpickling to the following.  When executing
> >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
> >>>
> >>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
> >>>                   ref^ := v.readRef(nelem.refType);
> >>>                 END;
> >>>
> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref).
> >>> CheckStoreTraced expects ref to be a normal reference to an object,
> >>> computes the address of its header (4 bytes less), and sets the
> >>> dirty bit.
> >>>
> >>> But ConvertPacking is reading in to a field of a recently created
> >>> object that is not first in the object.  It computes ref to point
> >>> to the field, and CheckStoreTraced actually steps on the previous
> >>> real field instead of the header.
> >>>
> >>> It looks like the actions of CheckStoreTraced are needed, but I
> >>> don't right off hand see how to recode Convert to fix this.  It's
> >>> use of ref can't be moved.  Presumably, there could be other unsafe
> >>> code elsewhere using the same technique to store data.
> >>>
> >>> Does the compiler need a different criterion on when to generate
> >>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
> >>> still is pointing inside a traced object.   Maybe people who modify
> >>> objects in this way need to explicitly call CheckStoreTraced?
> >>>
> >>> -- 
> >>> -------------------------------------------------------------
> >>> Rodney M. Bates, retired assistant professor
> >>> Dept. of Computer Science, Wichita State University
> >>> Wichita, KS 67260-0083
> >>> 316-978-3922
> >>> rodney.bates at wichita.edu
> >>> _______________________________________________
> >>> M3devel mailing list
> >>> M3devel at elegosoft.com
> >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> >>
> >> Antony Hosking                | Associate Professor
> >> Dept of Computer Science      | Office: +1 765 494-6001
> >> Purdue University             | Mobile: +1 765 427-5484
> >> 250 N. University Street      | Email:  hosking at cs.purdue.edu
> >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
> >>   _--_|\
> >> /      \
> >> \_.--._/    )
> >>        v    /
> >>
> >>
> >>
> >> _______________________________________________
> >> M3devel mailing list
> >> M3devel at elegosoft.com
> >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May  2 08:18:22 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 2 May 2007 02:18:22 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org>
References: <45970723.5070102@wichita.edu>
	<924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu>
	<1177226951.737.1.camel@faramir.m3w.org>
	<AEBE5122-F7F3-4CBA-823E-D8712E1AC419@cs.purdue.edu>
	<1178084752.8631.3.camel@faramir.m3w.org>
Message-ID: <F3AF7610-4655-4662-A530-ECCE833A1464@cs.purdue.edu>

Just grab CVS from before 2007/03/2007 (when I broke GC).  The  
pertinent fix was to RTTypeMap.m3, as at the current CVS head, which  
is dated 2007/01/09.

Thread/GC-related fixes are coming -- I know what the problem is but  
am in the middle of refactoring the heap allocation code.

On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote:

>   Is it possible to get this fix separatelly from threading fixes?  
> It's
> more important, esp since only program that crashes threading is
> TestThreads :)
>
>   TIA,
>   dd
>
> On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote:
>> Yes, this fix is in CVS head.
>>
>>
>> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote:
>>
>>> It looks like this fix never came through... With cvs-head CM3,
>>> unpickling of TextRefTbl.T instance still does not work.
>>>
>>> dd
>>>
>>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote:
>>>> Rodney,
>>>>
>>>> I've checked in fixes to RTTypeMap and clients that should restore
>>>> functional pickling support.  For some reason my commit messages  
>>>> are
>>>> awaiting moderator approval (it seems my commit id is not the  
>>>> same as
>>>> my subscription to the m3commit list) before they can be sent out.
>>>> Please make sure that unpickling now works for you.
>>>>
>>>> Best regards,
>>>>
>>>> Tony
>>>>
>>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote:
>>>>
>>>>> I tracked down a bug unpickling to the following.  When executing
>>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code:
>>>>>
>>>>>                 WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO
>>>>>                   ref^ := v.readRef(nelem.refType);
>>>>>                 END;
>>>>>
>>>>> has a compiler-generated call on RTCollector.CheckStoreTraced 
>>>>> (ref).
>>>>> CheckStoreTraced expects ref to be a normal reference to an  
>>>>> object,
>>>>> computes the address of its header (4 bytes less), and sets the
>>>>> dirty bit.
>>>>>
>>>>> But ConvertPacking is reading in to a field of a recently created
>>>>> object that is not first in the object.  It computes ref to point
>>>>> to the field, and CheckStoreTraced actually steps on the previous
>>>>> real field instead of the header.
>>>>>
>>>>> It looks like the actions of CheckStoreTraced are needed, but I
>>>>> don't right off hand see how to recode Convert to fix this.  It's
>>>>> use of ref can't be moved.  Presumably, there could be other  
>>>>> unsafe
>>>>> code elsewhere using the same technique to store data.
>>>>>
>>>>> Does the compiler need a different criterion on when to generate
>>>>> the call on CheckStoreTraced?  ref is an UNTRACED REF here, but it
>>>>> still is pointing inside a traced object.   Maybe people who  
>>>>> modify
>>>>> objects in this way need to explicitly call CheckStoreTraced?
>>>>>
>>>>> -- 
>>>>> -------------------------------------------------------------
>>>>> Rodney M. Bates, retired assistant professor
>>>>> Dept. of Computer Science, Wichita State University
>>>>> Wichita, KS 67260-0083
>>>>> 316-978-3922
>>>>> rodney.bates at wichita.edu
>>>>> _______________________________________________
>>>>> M3devel mailing list
>>>>> M3devel at elegosoft.com
>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>> Antony Hosking                | Associate Professor
>>>> Dept of Computer Science      | Office: +1 765 494-6001
>>>> Purdue University             | Mobile: +1 765 427-5484
>>>> 250 N. University Street      | Email:  hosking at cs.purdue.edu
>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking
>>>>   _--_|\
>>>> /      \
>>>> \_.--._/    )
>>>>        v    /
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From ronny.forberger at elegosoft.com  Wed May  2 11:24:14 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 02 May 2007 11:24:14 +0200
Subject: [M3devel] Just a test - please ignore
Message-ID: <463858BE.1040000@elegosoft.com>


-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From dragisha at m3w.org  Wed May  2 23:11:42 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Wed, 02 May 2007 23:11:42 +0200
Subject: [M3devel] formals initialization... NIL?
Message-ID: <1178140302.8631.26.camel@faramir.m3w.org>

99      PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args:
ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User }
=
100       BEGIN
101         IF args[FIRST(args)] = NIL THEN
102           RAISE User(
103             NEW(RefSeq.T).fromArray(
104               ARRAY OF REFANY { e.getRef(f), e.getQName()}
105             )
(m3gdb) p args
$1 = [16_00000001]

Is this normal? 16_00000001 is NIL? Why not printed as such?

-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May  5 09:44:07 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 05 May 2007 09:44:07 +0200
Subject: [M3devel] status of -linkall?
Message-ID: <1178351048.21091.3.camel@faramir.m3w.org>

I have a situation where -linkall works "as advertised", at least I
don't see anything missing yet :). Same set of modules, when included in
build of another program, fails to initialize...

If I've let anyone forget it :), it's plugin architecture where modules
plug to appropriate "connection points" simply by inclusion in
m3makefile.

Is it known problem with -linkall?

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Mon May  7 08:03:45 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Mon, 07 May 2007 08:03:45 +0200
Subject: [M3devel] import/exports loops... dynamic loading of Modula-3
	modules
Message-ID: <1178517825.3925.23.camel@faramir.m3w.org>

I am thinking about writing some loop detecting tool for (in)famous
order-of-initialization sindrome. Anyone did it already? I think we
really need something to help sort out of EXPORTS nightmare and it's
influence on RTLinker behaviour.

Also in news :), after I've tried to solve my problems with -linkall
(and got very big executabales as consequence, together with almost
working programs - at least until I try to make another similar build :)
- dynamic loading of modules was logical next step.

I've done it, and can confirm RTLinker.AddUnit works as advertised, any
time one calls it :) It also solves my plugin architecture problem, as
my plugin modules are not imported directly and thus their
initialization is fully performed with adequate RTLinker.AddUnit call.

If there's interest in how-to-make-dynamic-modules-happen, I can post it
somewhere. My google search did not show ay Modula-3 work on that.

dd
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Wed May  9 01:55:18 2007
From: darko at darko.org (Darko)
Date: Wed, 9 May 2007 01:55:18 +0200
Subject: [M3devel] m3.org expiry
Message-ID: <B588AC8B-D2DD-4271-B67D-4CC95C8E6F02@darko.org>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.



From wagner at elegosoft.com  Wed May  9 07:57:41 2007
From: wagner at elegosoft.com (Olaf Wagner)
Date: Wed, 9 May 2007 07:57:41 +0200
Subject: [M3devel] [darko@darko.org:  m3.org expiry]
Message-ID: <20070509055740.GA20162@elegosoft.com>

Hi Farshad,

I seem to remember that you tried to register m3.org for a period
longer than a year, but now it's gone again:

% host www.m3.org
www.m3.org is an alias for resalehost.networksolutions.com.
resalehost.networksolutions.com has address 205.178.189.128

What's the current state of things from your side? Should anybody
else get and host m3.org?

Olaf

----- Forwarded message from Darko <darko at darko.org> -----

X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com
To: Modula-3 developers <m3devel at elegosoft.com>
From: Darko <darko at darko.org>
Date: Wed, 9 May 2007 01:55:18 +0200
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed 
	version=3.0.3
X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de
X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de
X-Virus-Status: Clean
Subject: [M3devel] m3.org expiry
X-BeenThere: m3devel at elegosoft.com
X-Mailman-Version: 2.1.9
List-Id: <m3devel.elegosoft.com>
List-Unsubscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=unsubscribe>
List-Archive: <http://mail.elegosoft.com/pipermail/m3devel>
List-Post: <mailto:m3devel at elegosoft.com>
List-Help: <mailto:m3devel-request at elegosoft.com?subject=help>
List-Subscribe: <https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel>, 
	<mailto:m3devel-request at elegosoft.com?subject=subscribe>

It seems its time for the annual m3.org domain expiry. Is anyone in  
touch with Farshad or whoever owns the domain?

Darko.

_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel

----- End forwarded message -----

-- 
Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 23 45 869  fax: +49 30 23 45 86 95
http://www.elegosoft.com  |  Gesch?ftsf?hrer: Olaf Wagner  |   Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194


From hosking at cs.purdue.edu  Wed May 16 17:21:12 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:21:12 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <A79828E1-0986-4552-B0A1-C93811547613@cs.purdue.edu>

TestThreads is still locking up with 3000 threads (it works with 2000  
on my I386_DARWIN box).  I am now tracking this down.

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Wed May 16 17:19:54 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 16 May 2007 11:19:54 -0400
Subject: [M3devel] A GC/compiler/Pickle interaction bug
In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org>
References: <1177277134.737.47.camel@faramir.m3w.org>
	<9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu>
	<19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu>
	<1177281325.737.55.camel@faramir.m3w.org>
	<F32E2364-A890-4739-AC5F-D1E46616C503@cs.purdue.edu>
	<1177308751.737.63.camel@faramir.m3w.org>
Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu>

On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote:

> Thank you. I am using your posted compiler for now.
>
> dd
>
> On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote:
>> Something appears to be very broken with my most recent fixes for
>> your threading problems (since Apr 19, when I put the last cm3.gz
>> compiler bootstrap up).  I will try to track this down tomorrow.

I have tracked down and fixed this brokenness.  CVS head should now  
behave reasonably.

>>
>> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote:
>>
>>> Any pointer on how to do this?
>>>
>>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am
>>> using.
>>>
>>> dd
>>>
>>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote:
>>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt
>>>> for this fix to take effect.
>>>>
>>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote:
>>>>
>>>>> This works just fine for me on the most recent CVS head for
>>>>> I386_DARWIN.
>>>>>
>>>>> I wonder if you have not updated your compiler to match?  There  
>>>>> was
>>>>> a matching fix to the compiler.  You should build and ship:
>>>>>
>>>>> m3gc-simple
>>>>> m3core
>>>>> libm3
>>>>> m3middle
>>>>> m3linker
>>>>> m3front
>>>>> m3quake
>>>>> cm3
>>>>>
>>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your
>>>>> cm3.cfg).  This will get you a new compiler.
>>>>>
>>>>> Then, rebuild your CM3 installation from the CVS head.
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: Dragi?a Duri? <dragisha at m3w.org>
>>>>>> Date: April 22, 2007 5:25:33 PM EDT
>>>>>> To: Tony Hosking <hosking at cs.purdue.edu>
>>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug
>>>>>>
>>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote:
>>>>>>> Can you send me a stack dump from gdb at the point of the error?
>>>>>>> Even better, do you have a simple example program I can use to
>>>>>>> test
>>>>>>> with?
>>>>>>
>>>>>> Here is one. Quick and dirty, but it shows problem.
>>>>>>
>>>>>> -- 
>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>> <Pickles.m3>
>>>>>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Fri May 18 13:50:24 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Fri, 18 May 2007 13:50:24 +0200
Subject: [M3devel] Pickle compatibility...
Message-ID: <1179489025.6698.3.camel@faramir.m3w.org>

I have some data pickled by pm3 pickler... Now, reading these fails...
First app I've tried had online quiz high scores pickled, but now I have
to convert billing app :). What can I do to convert this?

dd

-- 
Dragi?a Duri? <dragisha at m3w.org>



From hosking at cs.purdue.edu  Sat May 19 22:00:36 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Sat, 19 May 2007 16:00:36 -0400
Subject: [M3devel] pthread in cm3...
In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>

I just checked in a patch that is closer to your code, and that lets  
me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
creation fails for that many).

On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:

> After implementing that workaround for result code 11 in that  
> SIGSUSPEND
> loop, every time during 1st or at most second pass (of 4000) it  
> stucks.
> Not same place every time, though... I think there are RCs in
> LockHeap/SuspendAll logic.
>
> dd
>
> On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
>> Yes, good thinking.  Tuning the threads systems is a good plan all
>> round.
>>
>> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
>>
>>> Just a random thought...
>>>
>>> I don't think my TestThreads is something special, but it's few  
>>> thread
>>> use patterns combined... And I've just had bright :) idea  
>>> yesterday -
>>> it's also decent benchmark for whole threading system... I think it
>>> would be nice to test it with 10000 rounds, 4000 threads each (once
>>> cm3
>>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
>>> tests for
>>> mine.
>>>
>>> I think these extra data structures and global locks in PTHREAD are
>>> big
>>> efficiency killers. Benchmark will show.
>>>
>>> dd
>>>
>>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
>>>> I take all of your points seriously.  One option would be to offer
>>>> your threads implementation as another build option for CM3.  I'm
>>>> going to track down the bug I introduced recently and then we can
>>>> consider how to move forward.
>>>>
>>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
>>>>
>>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
>>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Have been skimming through source of PTHREAD code... And I think
>>>>>>> job can
>>>>>>> be done without so much relying on how-they-did-it-before, esp
>>>>>>> with
>>>>>>> regard to list of waiters and similar internal and global
>>>>>>> structures.
>>>>>>> Also, I see number of global locks and I am sure they are
>>>>>>> congestion
>>>>>>> generators every now and while, esp in heavy threading  
>>>>>>> situations.
>>>>>>>
>>>>>>> Of course, there is number of approaches to this multi-thread
>>>>>>> situations. Mine being one of very nonconservative use of
>>>>>>> threads, I
>>>>>>> think it is important to remain open to possibly very big
>>>>>>> number of
>>>>>>> threads running in single process - meaning scalability is  
>>>>>>> one of
>>>>>>> primary objections... As global locks don't do well with
>>>>>>> scalability, I
>>>>>>> don't like "cm" and similar global congestion points.
>>>>>>
>>>>>> Yes, there are tensions between a thin/absent veneer between
>>>>>> language
>>>>>> threads and system threads.  Most important are issues of
>>>>>> preserving
>>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
>>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
>>>>>
>>>>> I know that paper, and being Modula-3 camp, I am - by definition -
>>>>> agreed to no-library-approach-for-threads :).
>>>>>
>>>>>>  There are
>>>>>> also questions of portability and debuggability.
>>>>>
>>>>>   Of course. That's why I am using only POSIX defined features,  
>>>>> and
>>>>> when
>>>>> in doubt - ones used by Boehm in his famous GC :).
>>>>>
>>>>>> I agree that global
>>>>>> locks are to be avoided where they cause known contention, but
>>>>>> there
>>>>>> are tradeoffs there too.
>>>>>
>>>>>   Global lock is bad, whatever reasons.
>>>>>
>>>>>> For large numbers of threads (as you appear
>>>>>> to need) I think we would need to adopt some other implementation
>>>>>> approach, possibly by multiplexing multiple lightweight user- 
>>>>>> level
>>>>>> threads on some smaller number of heavyweight system-level  
>>>>>> threads,
>>>>>> but then you run into scheduling and load-balancing problems.
>>>>>
>>>>>   I've argued this before... With O(1) process scheduling  
>>>>> available
>>>>> for
>>>>> four years from Linux, and in that time surely from everyone
>>>>> else... It
>>>>> would be bigger problem to maintain scheduling for special "BIG"
>>>>> cases
>>>>> inside our support libraries than to rely on operating system.  
>>>>> It's
>>>>> good
>>>>> that mainstream OS people recognized threads as Need, and it is
>>>>> time now
>>>>> for us to accept they did it well - AT LAST.
>>>>>
>>>>>   And, very important... I can't see what is heavyweight on system
>>>>> which
>>>>> does 10,000 context switches per second for 1500 threads with  
>>>>> 2% CPU
>>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
>>>>> WERE
>>>>> heavyweight four years ago, and they are not, long since. Even
>>>>> Windows
>>>>> has lightweight kernel-space threads :).
>>>>>
>>>>>>
>>>>>>> We talked about this at least once before, and I think I  
>>>>>>> remember
>>>>>>> you
>>>>>>> insisted on more compatibility than can be read from SPwM3.  
>>>>>>> Do you
>>>>>>> think
>>>>>>> best idea would be to integrate mine NPTL code into CM3 for
>>>>>>> people to
>>>>>>> trash and test, and let everyone select what is best for their
>>>>>>> situation?
>>>>>>
>>>>>> What I wanted was a situation where programs would be able to run
>>>>>> with the same tools (e.g., showheap, showthread) under both user-
>>>>>> level and system threading.  This goal has been achieved with the
>>>>>> current pthread-based implementation.
>>>>>
>>>>>   It is good reasoning, and it's one of reasons I did not suggest
>>>>> replacement... I think mine version is less bloated and I know  
>>>>> it's
>>>>> very
>>>>> good for long and stable process uptime we all expect from  
>>>>> Modula-3
>>>>> programs. But also, implied compatibilities outside of SPwM3 and
>>>>> direct
>>>>> demands from other parts of runtime were not on my list. These  
>>>>> I've
>>>>> respected, and it looks like these are good production time
>>>>> criteria. As
>>>>> opposed to excellent development time criteria you based yours on.
>>>>>
>>>>>> Moreover, I wanted something
>>>>>> where variations in thread support from one system to another  
>>>>>> could
>>>>>> be exploited most easily (such as for systems where thread  
>>>>>> suspend/
>>>>>> resume is provided as a primitive).  Again, the current
>>>>>> implementation achieves this, and runs with minimal target- 
>>>>>> specific
>>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
>>>>>> should be
>>>>>> relatively straightforward.
>>>>>
>>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
>>>>> extremmely
>>>>> POSIX, I see no problem.
>>>>>
>>>>>>
>>>>>>> Problems I had with my pthread implementation were all related
>>>>>>> to VM
>>>>>>> hell of earlier GC implementation... After you did that piece
>>>>>>> of art
>>>>>>> with new approach to GC, I expect infinite uptimes from my
>>>>>>> servers and
>>>>>>> bots :). Big thanks for that!
>>>>>>
>>>>>> Any native threading implementation is going to have problems  
>>>>>> with
>>>>>> VM-
>>>>>> based memory management.  I'm surprised to were able to get
>>>>>> anything
>>>>>> going with the VM-based GC.
>>>>>
>>>>>   Anything is pretty much - I have heavy multithreaded servers
>>>>> running
>>>>> literally for years,,, one of them is up since January of 2004, it
>>>>> services few hundreds of connected users (and up to 1500  
>>>>> threads) at
>>>>> almost every moment and breaks only when system reboots :). All  
>>>>> that
>>>>> with heavy integration of various C libraries.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> dd
>>>>>>> -- 
>>>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>>>
>>>>> -- 
>>>>> Dragi?a Duri? <dragisha at m3w.org>
>>>>
>>> -- 
>>> Dragi?a Duri? <dragisha at m3w.org>
>>
> -- 
> Dragi?a Duri? <dragisha at m3w.org>



From dragisha at m3w.org  Sat May 19 22:16:47 2007
From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=)
Date: Sat, 19 May 2007 22:16:47 +0200
Subject: [M3devel] pthread in cm3...
In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
References: <1177253233.737.18.camel@faramir.m3w.org>
	<4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu>
	<1177316476.737.97.camel@faramir.m3w.org>
	<BE96F94A-1ED7-4D73-9481-5CBC6A1C74BF@cs.purdue.edu>
	<1177484112.4826.9.camel@faramir.m3w.org>
	<B7970275-5172-457E-9783-A0F837984C99@cs.purdue.edu>
	<1177518937.15060.6.camel@faramir.m3w.org>
	<0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu>
Message-ID: <1179605808.8468.4.camel@faramir.m3w.org>

I am running 4000, but it's not most important anyway...

Did you see that (Un)LockMutex semantics I've enabled so one can
overload methods invoked by LOCK .. DO?

dd

On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote:
> I just checked in a patch that is closer to your code, and that lets  
> me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread  
> creation fails for that many).
> 
> On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote:
> 
> > After implementing that workaround for result code 11 in that  
> > SIGSUSPEND
> > loop, every time during 1st or at most second pass (of 4000) it  
> > stucks.
> > Not same place every time, though... I think there are RCs in
> > LockHeap/SuspendAll logic.
> >
> > dd
> >
> > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote:
> >> Yes, good thinking.  Tuning the threads systems is a good plan all
> >> round.
> >>
> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote:
> >>
> >>> Just a random thought...
> >>>
> >>> I don't think my TestThreads is something special, but it's few  
> >>> thread
> >>> use patterns combined... And I've just had bright :) idea  
> >>> yesterday -
> >>> it's also decent benchmark for whole threading system... I think it
> >>> would be nice to test it with 10000 rounds, 4000 threads each (once
> >>> cm3
> >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do
> >>> tests for
> >>> mine.
> >>>
> >>> I think these extra data structures and global locks in PTHREAD are
> >>> big
> >>> efficiency killers. Benchmark will show.
> >>>
> >>> dd
> >>>
> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote:
> >>>> I take all of your points seriously.  One option would be to offer
> >>>> your threads implementation as another build option for CM3.  I'm
> >>>> going to track down the bug I introduced recently and then we can
> >>>> consider how to move forward.
> >>>>
> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote:
> >>>>
> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote:
> >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote:
> >>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> Have been skimming through source of PTHREAD code... And I think
> >>>>>>> job can
> >>>>>>> be done without so much relying on how-they-did-it-before, esp
> >>>>>>> with
> >>>>>>> regard to list of waiters and similar internal and global
> >>>>>>> structures.
> >>>>>>> Also, I see number of global locks and I am sure they are
> >>>>>>> congestion
> >>>>>>> generators every now and while, esp in heavy threading  
> >>>>>>> situations.
> >>>>>>>
> >>>>>>> Of course, there is number of approaches to this multi-thread
> >>>>>>> situations. Mine being one of very nonconservative use of
> >>>>>>> threads, I
> >>>>>>> think it is important to remain open to possibly very big
> >>>>>>> number of
> >>>>>>> threads running in single process - meaning scalability is  
> >>>>>>> one of
> >>>>>>> primary objections... As global locks don't do well with
> >>>>>>> scalability, I
> >>>>>>> don't like "cm" and similar global congestion points.
> >>>>>>
> >>>>>> Yes, there are tensions between a thin/absent veneer between
> >>>>>> language
> >>>>>> threads and system threads.  Most important are issues of
> >>>>>> preserving
> >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper
> >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042).
> >>>>>
> >>>>> I know that paper, and being Modula-3 camp, I am - by definition -
> >>>>> agreed to no-library-approach-for-threads :).
> >>>>>
> >>>>>>  There are
> >>>>>> also questions of portability and debuggability.
> >>>>>
> >>>>>   Of course. That's why I am using only POSIX defined features,  
> >>>>> and
> >>>>> when
> >>>>> in doubt - ones used by Boehm in his famous GC :).
> >>>>>
> >>>>>> I agree that global
> >>>>>> locks are to be avoided where they cause known contention, but
> >>>>>> there
> >>>>>> are tradeoffs there too.
> >>>>>
> >>>>>   Global lock is bad, whatever reasons.
> >>>>>
> >>>>>> For large numbers of threads (as you appear
> >>>>>> to need) I think we would need to adopt some other implementation
> >>>>>> approach, possibly by multiplexing multiple lightweight user- 
> >>>>>> level
> >>>>>> threads on some smaller number of heavyweight system-level  
> >>>>>> threads,
> >>>>>> but then you run into scheduling and load-balancing problems.
> >>>>>
> >>>>>   I've argued this before... With O(1) process scheduling  
> >>>>> available
> >>>>> for
> >>>>> four years from Linux, and in that time surely from everyone
> >>>>> else... It
> >>>>> would be bigger problem to maintain scheduling for special "BIG"
> >>>>> cases
> >>>>> inside our support libraries than to rely on operating system.  
> >>>>> It's
> >>>>> good
> >>>>> that mainstream OS people recognized threads as Need, and it is
> >>>>> time now
> >>>>> for us to accept they did it well - AT LAST.
> >>>>>
> >>>>>   And, very important... I can't see what is heavyweight on system
> >>>>> which
> >>>>> does 10,000 context switches per second for 1500 threads with  
> >>>>> 2% CPU
> >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads  
> >>>>> WERE
> >>>>> heavyweight four years ago, and they are not, long since. Even
> >>>>> Windows
> >>>>> has lightweight kernel-space threads :).
> >>>>>
> >>>>>>
> >>>>>>> We talked about this at least once before, and I think I  
> >>>>>>> remember
> >>>>>>> you
> >>>>>>> insisted on more compatibility than can be read from SPwM3.  
> >>>>>>> Do you
> >>>>>>> think
> >>>>>>> best idea would be to integrate mine NPTL code into CM3 for
> >>>>>>> people to
> >>>>>>> trash and test, and let everyone select what is best for their
> >>>>>>> situation?
> >>>>>>
> >>>>>> What I wanted was a situation where programs would be able to run
> >>>>>> with the same tools (e.g., showheap, showthread) under both user-
> >>>>>> level and system threading.  This goal has been achieved with the
> >>>>>> current pthread-based implementation.
> >>>>>
> >>>>>   It is good reasoning, and it's one of reasons I did not suggest
> >>>>> replacement... I think mine version is less bloated and I know  
> >>>>> it's
> >>>>> very
> >>>>> good for long and stable process uptime we all expect from  
> >>>>> Modula-3
> >>>>> programs. But also, implied compatibilities outside of SPwM3 and
> >>>>> direct
> >>>>> demands from other parts of runtime were not on my list. These  
> >>>>> I've
> >>>>> respected, and it looks like these are good production time
> >>>>> criteria. As
> >>>>> opposed to excellent development time criteria you based yours on.
> >>>>>
> >>>>>> Moreover, I wanted something
> >>>>>> where variations in thread support from one system to another  
> >>>>>> could
> >>>>>> be exploited most easily (such as for systems where thread  
> >>>>>> suspend/
> >>>>>> resume is provided as a primitive).  Again, the current
> >>>>>> implementation achieves this, and runs with minimal target- 
> >>>>>> specific
> >>>>>> code on Darwin, Solaris, and Linux.  Ports to other targets
> >>>>>> should be
> >>>>>> relatively straightforward.
> >>>>>
> >>>>>   I've not ported mine outside of LINUXLIBC6, but as it's  
> >>>>> extremmely
> >>>>> POSIX, I see no problem.
> >>>>>
> >>>>>>
> >>>>>>> Problems I had with my pthread implementation were all related
> >>>>>>> to VM
> >>>>>>> hell of earlier GC implementation... After you did that piece
> >>>>>>> of art
> >>>>>>> with new approach to GC, I expect infinite uptimes from my
> >>>>>>> servers and
> >>>>>>> bots :). Big thanks for that!
> >>>>>>
> >>>>>> Any native threading implementation is going to have problems  
> >>>>>> with
> >>>>>> VM-
> >>>>>> based memory management.  I'm surprised to were able to get
> >>>>>> anything
> >>>>>> going with the VM-based GC.
> >>>>>
> >>>>>   Anything is pretty much - I have heavy multithreaded servers
> >>>>> running
> >>>>> literally for years,,, one of them is up since January of 2004, it
> >>>>> services few hundreds of connected users (and up to 1500  
> >>>>> threads) at
> >>>>> almost every moment and breaks only when system reboots :). All  
> >>>>> that
> >>>>> with heavy integration of various C libraries.
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> dd
> >>>>>>> -- 
> >>>>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>>>
> >>>>> -- 
> >>>>> Dragi?a Duri? <dragisha at m3w.org>
> >>>>
> >>> -- 
> >>> Dragi?a Duri? <dragisha at m3w.org>
> >>
> > -- 
> > Dragi?a Duri? <dragisha at m3w.org>
> 
-- 
Dragi?a Duri? <dragisha at m3w.org>



From darko at darko.org  Tue May 22 15:13:09 2007
From: darko at darko.org (Darko)
Date: Tue, 22 May 2007 15:13:09 +0200
Subject: [M3devel] Darwin Intel Data Alignment
Message-ID: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>

Hi,

I'm having problems trying to use byte aligned fields in structures  
in the new Darwin Intel compiler. My understanding is that the  
processor accepts accesses on (word, etc) data not aligned on their  
normal boundaries. Is there something in the back end which means  
this is not possible?

The <*LAZYALIGN*> pragma doesn't work properly and I get:

"../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
unaligned store_indirect  type=11  s/a=32/16
"../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
to find integer type?  type=Int.32  s/o/a=32/64/16

when I compile. I think the platform independent alignment  
calculation code is correct, so I'm thinking this is a back end  
limitation.

Any thoughts?


Cheers,
Darko.



From hosking at cs.purdue.edu  Tue May 22 17:22:55 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Tue, 22 May 2007 11:22:55 -0400
Subject: [M3devel] Darwin Intel Data Alignment
In-Reply-To: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
References: <E45F925C-E139-4A7E-A0CA-5993B2574EC1@darko.org>
Message-ID: <ACCD1401-62C4-4E4A-AD00-CFF550072A91@cs.purdue.edu>

Hmm.  I am not familiar with this aspect of things.  One thing would  
be to check if things work for you on LINUXLIBC6.  Then we just need  
to figure out what is different between the Darwin and Linux Intel  
targets.  Can you send me some test code that demonstrates the problem?

On May 22, 2007, at 9:13 AM, Darko wrote:

> Hi,
>
> I'm having problems trying to use byte aligned fields in structures  
> in the new Darwin Intel compiler. My understanding is that the  
> processor accepts accesses on (word, etc) data not aligned on their  
> normal boundaries. Is there something in the back end which means  
> this is not possible?
>
> The <*LAZYALIGN*> pragma doesn't work properly and I get:
>
> "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR ***  
> unaligned store_indirect  type=11  s/a=32/16
> "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable  
> to find integer type?  type=Int.32  s/o/a=32/64/16
>
> when I compile. I think the platform independent alignment  
> calculation code is correct, so I'm thinking this is a back end  
> limitation.
>
> Any thoughts?
>
>
> Cheers,
> Darko.



From rodney.bates at wichita.edu  Sat May 26 23:48:17 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sat, 26 May 2007 16:48:17 -0500
Subject: [M3devel] Still can't bootstrap CM3
Message-ID: <4658AB21.5030100@wichita.edu>

I still can't get a working cm3 built. I tried the following:

Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz
and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there.

Did 'cvs -z3 update -dAP' in the top of my copy of cm3.

In scripts, did 'do-cm3-min.sh buildship', which ended with:

-----------------------------------------------------------------------------
=== package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  && cm3 -ship 
-DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
--- building in LINUXLIBC6 ---

ignoring ../src/m3overrides

new source -> compiling ConvertPacking.m3
new "ConvertPacking.mo" -> archiving libm3.a

Fatal Error: incomplete library

ThreadPosix.i3: missing object types:  _t1541f475
imported by:  TimeStamp.m3
  *** execution of  failed ***

------------------------------------------------------------------------------

I believe this is the same symptom I had the last time I tried.

-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From rodney.bates at wichita.edu  Sun May 27 18:23:51 2007
From: rodney.bates at wichita.edu (Rodney M. Bates)
Date: Sun, 27 May 2007 11:23:51 -0500
Subject: [M3devel] Can't load dynamically linked m3bundle
Message-ID: <4659B097.5040908@wichita.edu>

While trying to bootstrap CM3, I get this:

-----------------------------------------------------------------------------------------------------
rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle
-rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle
rodney at runnymede:/usr/local/cm3/bin$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically 
linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle
-bash: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle
/usr/bin/ldd: line 125: ./m3bundle: No such file or directory
rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue
Mandriva Linux release 2007.0 (Cooker) for i586
Kernel 2.6.17-5mdv on an i686 / \l
rodney at runnymede:/usr/local/cm3/bin$
------------------------------------------------------------------------------------------------------

On an earlier Mandriva release, m3bundle turns out statically linked.  In both cases,
the m3makefile for m3bundle contains 'build_standalone ( )'.

------------------------------------------------------------------------------------------------------
[rodney at selkirk bin]$ pwd
/usr/local/cm3/bin
[rodney at selkirk bin]$ file m3bundle
m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically 
linked, not stripped
[rodney at selkirk bin]$ cat /etc/issue
Mandrakelinux release 10.2 (Limited Edition 2005) for i586
Kernel 2.6.11-6mdk on an i686 / \l
[rodney at selkirk bin]$
------------------------------------------------------------------------------------------------------

I am guessing that there is some dynamic library that m3bundle wants that I don't have
installed, but how can I find it?  'objdump -T m3bundle' gives a lot of entries, the
most interesting ones naming GCC_3.0.  The system where it fails has gcc 4.1.1 installed.
I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a
symbolic link to it, but to no avail.


-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu


From hosking at cs.purdue.edu  Mon May 28 18:55:21 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Mon, 28 May 2007 12:55:21 -0400
Subject: [M3devel] Still can't bootstrap CM3
In-Reply-To: <4658AB21.5030100@wichita.edu>
References: <4658AB21.5030100@wichita.edu>
Message-ID: <D880263B-B69A-4A92-BE10-1E5A3F940355@cs.purdue.edu>

Did you do realclean before buildship?

I have successfully bootstrapped from a clean CVS head using that cm3  
binary on several Linux platforms.

On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote:

> I still can't get a working cm3 built. I tried the following:
>
> Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ 
> pub/hosking/m3/LINUXLIBC6/cm3.gz
> and put it in /usr/local/cm3/bin, which is on my path and the  
> command 'cm3' finds it there.
>
> Did 'cvs -z3 update -dAP' in the top of my copy of cm3.
>
> In scripts, did 'do-cm3-min.sh buildship', which ended with:
>
> ---------------------------------------------------------------------- 
> -------
> === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 ===
>  +++ cm3 -build  -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  &&  
> cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
> new source -> compiling ConvertPacking.m3
> new "ConvertPacking.mo" -> archiving libm3.a
>
> Fatal Error: incomplete library
>
> ThreadPosix.i3: missing object types:  _t1541f475
> imported by:  TimeStamp.m3
>  *** execution of  failed ***
>
> ---------------------------------------------------------------------- 
> --------
>
> I believe this is the same symptom I had the last time I tried.
>
> -- 
> -------------------------------------------------------------
> Rodney M. Bates, retired assistant professor
> Dept. of Computer Science, Wichita State University
> Wichita, KS 67260-0083
> 316-978-3922
> rodney.bates at wichita.edu



From ronny.forberger at elegosoft.com  Wed May 30 15:24:16 2007
From: ronny.forberger at elegosoft.com (Ronny Forberger)
Date: Wed, 30 May 2007 15:24:16 +0200
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
Message-ID: <465D7B00.8000205@elegosoft.com>

Dear m3-Developers,

I could not successfully build the cm3 compiler when invoking

   

> ./do-cm3-std.sh buildship


from the script directory as described in the documentation.

The build process stops by the following:

> === package /root/cm3/m3-sys/m3cgcat ===
>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/cm3'  +++
> --- building in LINUXLIBC6 ---
>
> ignoring ../src/m3overrides
>
>  -> linking m3cgcat
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InnerLockMutex':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined 
> reference to `pthread_cond_wait'
>
[ some similar lines removed ]

>
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PushEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined 
> reference to `pthread_getspecific'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `RTHooks__PopEFrame':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined 
> reference to `pthread_setspecific'
> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In 
> function `ThreadPThread__InitHandlers':
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined 
> reference to `pthread_key_create'
> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined 
> reference to `pthread_setspecific'
> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In 
> function `vfork':
> (.text+0x1): undefined reference to `pthread_create'
> collect2: ld returned 1 exit status
> Fatal Error: package build failed
>  *** execution of  failed ***


Looks like gcc cannot link against the pthread-lib.

I have already added

> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]

to the SYSTEM_LIBS array in cm3.cfg.

This is a Debian 4.0  'etch' system (x86) where libpthread.a resides in 
/usr/lib.

I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread 
arccordingly which become linked against libpthread.a properly.


Do you have any glues?

Best regards,

Ronny

-- 
Ronny Forberger
Systemadministrator

elego Software Solutions GmbH           ronny.forberger(at)elegosoft.com
Ohmstra?e 9,  10179 Berlin              HRB 77719
Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner

Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de .



From hosking at cs.purdue.edu  Wed May 30 16:47:56 2007
From: hosking at cs.purdue.edu (Tony Hosking)
Date: Wed, 30 May 2007 10:47:56 -0400
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
References: <465D7B00.8000205@elegosoft.com>
Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu>

Can you use the -commands option to cm3 to show the link command?  It  
certainly looks like it is not finding the pthread library.

On May 30, 2007, at 9:24 AM, Ronny Forberger wrote:

> Dear m3-Developers,
>
> I could not successfully build the cm3 compiler when invoking
>
>
>> ./do-cm3-std.sh buildship
>
>
> from the script directory as described in the documentation.
>
> The build process stops by the following:
>
>> === package /root/cm3/m3-sys/m3cgcat ===
>>  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship -DROOT='/root/ 
>> cm3'  +++
>> --- building in LINUXLIBC6 ---
>>
>> ignoring ../src/m3overrides
>>
>>  -> linking m3cgcat
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:  
>> undefined reference to `pthread_cond_wait'
>>
> [ some similar lines removed ]
>
>>
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PushEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:  
>> undefined reference to `pthread_getspecific'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `RTHooks__PopEFrame':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:  
>> undefined reference to `pthread_setspecific'
>> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a 
>> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers':
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:  
>> undefined reference to `pthread_key_create'
>> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:  
>> undefined reference to `pthread_setspecific'
>> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):  
>> In function `vfork':
>> (.text+0x1): undefined reference to `pthread_create'
>> collect2: ld returned 1 exit status
>> Fatal Error: package build failed
>>  *** execution of  failed ***
>
>
> Looks like gcc cannot link against the pthread-lib.
>
> I have already added
>
>> "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
>
> to the SYSTEM_LIBS array in cm3.cfg.
>
> This is a Debian 4.0  'etch' system (x86) where libpthread.a  
> resides in /usr/lib.
>
> I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread  
> arccordingly which become linked against libpthread.a properly.
>
>
> Do you have any glues?
>
> Best regards,
>
> Ronny
>
> -- 
> Ronny Forberger
> Systemadministrator
>
> elego Software Solutions GmbH           ronny.forberger(at) 
> elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der Gesellschaft: Berlin
> http://www.elegosoft.com                Gesch?ftsf?hrer: Olaf Wagner
>
> Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) 
> elego.de .



From dabenavidesd at yahoo.es  Thu May 31 01:42:04 2007
From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.)
Date: Thu, 31 May 2007 01:42:04 +0200 (CEST)
Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap
In-Reply-To: <465D7B00.8000205@elegosoft.com>
Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com>

Hi,
Looks that you haven't used the SYSTEM_LIBORDER array,
must be "PTHREAD" there in order to really link it.

Maybe this little guide can help. Can have several
omissions, but It works well in (K)ubuntu 6.06 and
6.10 
http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000


--- Ronny Forberger <ronny.forberger at elegosoft.com>
wrote:
> Dear m3-Developers,
> 
> I could not successfully build the cm3 compiler when
> invoking
> 
>    
> 
> > ./do-cm3-std.sh buildship
> 
> 
> from the script directory as described in the
> documentation.
> 
> The build process stops by the following:
> 
> > === package /root/cm3/m3-sys/m3cgcat ===
> >  +++ cm3 -build  -DROOT='/root/cm3'  && cm3 -ship
> -DROOT='/root/cm3'  +++
> > --- building in LINUXLIBC6 ---
> >
> > ignoring ../src/m3overrides
> >
> >  -> linking m3cgcat
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InnerLockMutex':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142:
> undefined 
> > reference to `pthread_cond_wait'
> >
> [ some similar lines removed ]
> 
> >
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PushEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429:
> undefined 
> > reference to `pthread_getspecific'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `RTHooks__PopEFrame':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo):
> In 
> > function `ThreadPThread__InitHandlers':
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442:
> undefined 
> > reference to `pthread_key_create'
> >
>
/root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443:
> undefined 
> > reference to `pthread_setspecific'
> >
>
/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o):
> In 
> > function `vfork':
> > (.text+0x1): undefined reference to
> `pthread_create'
> > collect2: ld returned 1 exit status
> > Fatal Error: package build failed
> >  *** execution of  failed ***
> 
> 
> Looks like gcc cannot link against the pthread-lib.
> 
> I have already added
> 
> > "PTHREAD"    : [ "-L/usr/lib", "-lpthread" ]
> 
> to the SYSTEM_LIBS array in cm3.cfg.
> 
> This is a Debian 4.0  'etch' system (x86) where
> libpthread.a resides in 
> /usr/lib.
> 
> I can build trivial C programs like gcc foo.c -L
> /usr/lib -lpthread 
> arccordingly which become linked against
> libpthread.a properly.
> 
> 
> Do you have any glues?
> 
> Best regards,
> 
> Ronny
> 
> -- 
> Ronny Forberger
> Systemadministrator
> 
> elego Software Solutions GmbH          
> ronny.forberger(at)elegosoft.com
> Ohmstra?e 9,  10179 Berlin              HRB 77719
> Tel.: +49 30 23 45 86 96                Amtsgericht
> Charlottenburg
> Fax:  +49 30 23 45 86 95                Sitz der
> Gesellschaft: Berlin
> http://www.elegosoft.com               
> Gesch?ftsf?hrer: Olaf Wagner
> 
> Bitte senden Sie Ihre technischen Support-Anfragen
> an admins(at)elego.de .
> 
> 



		
______________________________________________ 
LLama Gratis a cualquier PC del Mundo. 
Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. 
http://es.voice.yahoo.com