From dragisha at m3w.org Wed May 2 07:45:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 07:45:52 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: <1178084752.8631.3.camel@faramir.m3w.org> Is it possible to get this fix separatelly from threading fixes? It's more important, esp since only program that crashes threading is TestThreads :) TIA, dd On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: > Yes, this fix is in CVS head. > > > On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > > > It looks like this fix never came through... With cvs-head CM3, > > unpickling of TextRefTbl.T instance still does not work. > > > > dd > > > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > >> Rodney, > >> > >> I've checked in fixes to RTTypeMap and clients that should restore > >> functional pickling support. For some reason my commit messages are > >> awaiting moderator approval (it seems my commit id is not the same as > >> my subscription to the m3commit list) before they can be sent out. > >> Please make sure that unpickling now works for you. > >> > >> Best regards, > >> > >> Tony > >> > >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > >> > >>> I tracked down a bug unpickling to the following. When executing > >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > >>> > >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > >>> ref^ := v.readRef(nelem.refType); > >>> END; > >>> > >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > >>> CheckStoreTraced expects ref to be a normal reference to an object, > >>> computes the address of its header (4 bytes less), and sets the > >>> dirty bit. > >>> > >>> But ConvertPacking is reading in to a field of a recently created > >>> object that is not first in the object. It computes ref to point > >>> to the field, and CheckStoreTraced actually steps on the previous > >>> real field instead of the header. > >>> > >>> It looks like the actions of CheckStoreTraced are needed, but I > >>> don't right off hand see how to recode Convert to fix this. It's > >>> use of ref can't be moved. Presumably, there could be other unsafe > >>> code elsewhere using the same technique to store data. > >>> > >>> Does the compiler need a different criterion on when to generate > >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > >>> still is pointing inside a traced object. Maybe people who modify > >>> objects in this way need to explicitly call CheckStoreTraced? > >>> > >>> -- > >>> ------------------------------------------------------------- > >>> Rodney M. Bates, retired assistant professor > >>> Dept. of Computer Science, Wichita State University > >>> Wichita, KS 67260-0083 > >>> 316-978-3922 > >>> rodney.bates at wichita.edu > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> Antony Hosking | Associate Professor > >> Dept of Computer Science | Office: +1 765 494-6001 > >> Purdue University | Mobile: +1 765 427-5484 > >> 250 N. University Street | Email: hosking at cs.purdue.edu > >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > >> _--_|\ > >> / \ > >> \_.--._/ ) > >> v / > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Wed May 2 08:18:22 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 2 May 2007 02:18:22 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> <1178084752.8631.3.camel@faramir.m3w.org> Message-ID: Just grab CVS from before 2007/03/2007 (when I broke GC). The pertinent fix was to RTTypeMap.m3, as at the current CVS head, which is dated 2007/01/09. Thread/GC-related fixes are coming -- I know what the problem is but am in the middle of refactoring the heap allocation code. On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote: > Is it possible to get this fix separatelly from threading fixes? > It's > more important, esp since only program that crashes threading is > TestThreads :) > > TIA, > dd > > On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: >> Yes, this fix is in CVS head. >> >> >> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: >> >>> It looks like this fix never came through... With cvs-head CM3, >>> unpickling of TextRefTbl.T instance still does not work. >>> >>> dd >>> >>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >>>> Rodney, >>>> >>>> I've checked in fixes to RTTypeMap and clients that should restore >>>> functional pickling support. For some reason my commit messages >>>> are >>>> awaiting moderator approval (it seems my commit id is not the >>>> same as >>>> my subscription to the m3commit list) before they can be sent out. >>>> Please make sure that unpickling now works for you. >>>> >>>> Best regards, >>>> >>>> Tony >>>> >>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >>>> >>>>> I tracked down a bug unpickling to the following. When executing >>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>>>> >>>>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>>>> ref^ := v.readRef(nelem.refType); >>>>> END; >>>>> >>>>> has a compiler-generated call on RTCollector.CheckStoreTraced >>>>> (ref). >>>>> CheckStoreTraced expects ref to be a normal reference to an >>>>> object, >>>>> computes the address of its header (4 bytes less), and sets the >>>>> dirty bit. >>>>> >>>>> But ConvertPacking is reading in to a field of a recently created >>>>> object that is not first in the object. It computes ref to point >>>>> to the field, and CheckStoreTraced actually steps on the previous >>>>> real field instead of the header. >>>>> >>>>> It looks like the actions of CheckStoreTraced are needed, but I >>>>> don't right off hand see how to recode Convert to fix this. It's >>>>> use of ref can't be moved. Presumably, there could be other >>>>> unsafe >>>>> code elsewhere using the same technique to store data. >>>>> >>>>> Does the compiler need a different criterion on when to generate >>>>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>>>> still is pointing inside a traced object. Maybe people who >>>>> modify >>>>> objects in this way need to explicitly call CheckStoreTraced? >>>>> >>>>> -- >>>>> ------------------------------------------------------------- >>>>> Rodney M. Bates, retired assistant professor >>>>> Dept. of Computer Science, Wichita State University >>>>> Wichita, KS 67260-0083 >>>>> 316-978-3922 >>>>> rodney.bates at wichita.edu >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Science | Office: +1 765 494-6001 >>>> Purdue University | Mobile: +1 765 427-5484 >>>> 250 N. University Street | Email: hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From ronny.forberger at elegosoft.com Wed May 2 11:24:14 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 02 May 2007 11:24:14 +0200 Subject: [M3devel] Just a test - please ignore Message-ID: <463858BE.1040000@elegosoft.com> -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From dragisha at m3w.org Wed May 2 23:11:42 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 23:11:42 +0200 Subject: [M3devel] formals initialization... NIL? Message-ID: <1178140302.8631.26.camel@faramir.m3w.org> 99 PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args: ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User } = 100 BEGIN 101 IF args[FIRST(args)] = NIL THEN 102 RAISE User( 103 NEW(RefSeq.T).fromArray( 104 ARRAY OF REFANY { e.getRef(f), e.getQName()} 105 ) (m3gdb) p args $1 = [16_00000001] Is this normal? 16_00000001 is NIL? Why not printed as such? -- Dragi?a Duri? From dragisha at m3w.org Sat May 5 09:44:07 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 05 May 2007 09:44:07 +0200 Subject: [M3devel] status of -linkall? Message-ID: <1178351048.21091.3.camel@faramir.m3w.org> I have a situation where -linkall works "as advertised", at least I don't see anything missing yet :). Same set of modules, when included in build of another program, fails to initialize... If I've let anyone forget it :), it's plugin architecture where modules plug to appropriate "connection points" simply by inclusion in m3makefile. Is it known problem with -linkall? dd -- Dragi?a Duri? From dragisha at m3w.org Mon May 7 08:03:45 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 07 May 2007 08:03:45 +0200 Subject: [M3devel] import/exports loops... dynamic loading of Modula-3 modules Message-ID: <1178517825.3925.23.camel@faramir.m3w.org> I am thinking about writing some loop detecting tool for (in)famous order-of-initialization sindrome. Anyone did it already? I think we really need something to help sort out of EXPORTS nightmare and it's influence on RTLinker behaviour. Also in news :), after I've tried to solve my problems with -linkall (and got very big executabales as consequence, together with almost working programs - at least until I try to make another similar build :) - dynamic loading of modules was logical next step. I've done it, and can confirm RTLinker.AddUnit works as advertised, any time one calls it :) It also solves my plugin architecture problem, as my plugin modules are not imported directly and thus their initialization is fully performed with adequate RTLinker.AddUnit call. If there's interest in how-to-make-dynamic-modules-happen, I can post it somewhere. My google search did not show ay Modula-3 work on that. dd -- Dragi?a Duri? From darko at darko.org Wed May 9 01:55:18 2007 From: darko at darko.org (Darko) Date: Wed, 9 May 2007 01:55:18 +0200 Subject: [M3devel] m3.org expiry Message-ID: It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. From wagner at elegosoft.com Wed May 9 07:57:41 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 9 May 2007 07:57:41 +0200 Subject: [M3devel] [darko@darko.org: m3.org expiry] Message-ID: <20070509055740.GA20162@elegosoft.com> Hi Farshad, I seem to remember that you tried to register m3.org for a period longer than a year, but now it's gone again: % host www.m3.org www.m3.org is an alias for resalehost.networksolutions.com. resalehost.networksolutions.com has address 205.178.189.128 What's the current state of things from your side? Should anybody else get and host m3.org? Olaf ----- Forwarded message from Darko ----- X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com To: Modula-3 developers From: Darko Date: Wed, 9 May 2007 01:55:18 +0200 X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed version=3.0.3 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Subject: [M3devel] m3.org expiry X-BeenThere: m3devel at elegosoft.com X-Mailman-Version: 2.1.9 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed May 16 17:21:12 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:21:12 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: TestThreads is still locking up with 3000 threads (it works with 2000 on my I386_DARWIN box). I am now tracking this down. On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed May 16 17:19:54 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:19:54 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> <1177281325.737.55.camel@faramir.m3w.org> <1177308751.737.63.camel@faramir.m3w.org> Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu> On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote: > Thank you. I am using your posted compiler for now. > > dd > > On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote: >> Something appears to be very broken with my most recent fixes for >> your threading problems (since Apr 19, when I put the last cm3.gz >> compiler bootstrap up). I will try to track this down tomorrow. I have tracked down and fixed this brokenness. CVS head should now behave reasonably. >> >> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote: >> >>> Any pointer on how to do this? >>> >>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am >>> using. >>> >>> dd >>> >>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote: >>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt >>>> for this fix to take effect. >>>> >>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: >>>> >>>>> This works just fine for me on the most recent CVS head for >>>>> I386_DARWIN. >>>>> >>>>> I wonder if you have not updated your compiler to match? There >>>>> was >>>>> a matching fix to the compiler. You should build and ship: >>>>> >>>>> m3gc-simple >>>>> m3core >>>>> libm3 >>>>> m3middle >>>>> m3linker >>>>> m3front >>>>> m3quake >>>>> cm3 >>>>> >>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your >>>>> cm3.cfg). This will get you a new compiler. >>>>> >>>>> Then, rebuild your CM3 installation from the CVS head. >>>>> >>>>> Begin forwarded message: >>>>> >>>>>> From: Dragi?a Duri? >>>>>> Date: April 22, 2007 5:25:33 PM EDT >>>>>> To: Tony Hosking >>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >>>>>> >>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>>>>>> Can you send me a stack dump from gdb at the point of the error? >>>>>>> Even better, do you have a simple example program I can use to >>>>>>> test >>>>>>> with? >>>>>> >>>>>> Here is one. Quick and dirty, but it shows problem. >>>>>> >>>>>> -- >>>>>> Dragi?a Duri? >>>>>> >>>>> >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Fri May 18 13:50:24 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 18 May 2007 13:50:24 +0200 Subject: [M3devel] Pickle compatibility... Message-ID: <1179489025.6698.3.camel@faramir.m3w.org> I have some data pickled by pm3 pickler... Now, reading these fails... First app I've tried had online quiz high scores pickled, but now I have to convert billing app :). What can I do to convert this? dd -- Dragi?a Duri? From hosking at cs.purdue.edu Sat May 19 22:00:36 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 19 May 2007 16:00:36 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> I just checked in a patch that is closer to your code, and that lets me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread creation fails for that many). On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Sat May 19 22:16:47 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 19 May 2007 22:16:47 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> Message-ID: <1179605808.8468.4.camel@faramir.m3w.org> I am running 4000, but it's not most important anyway... Did you see that (Un)LockMutex semantics I've enabled so one can overload methods invoked by LOCK .. DO? dd On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote: > I just checked in a patch that is closer to your code, and that lets > me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread > creation fails for that many). > > On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > > > After implementing that workaround for result code 11 in that > > SIGSUSPEND > > loop, every time during 1st or at most second pass (of 4000) it > > stucks. > > Not same place every time, though... I think there are RCs in > > LockHeap/SuspendAll logic. > > > > dd > > > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > >> Yes, good thinking. Tuning the threads systems is a good plan all > >> round. > >> > >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > >> > >>> Just a random thought... > >>> > >>> I don't think my TestThreads is something special, but it's few > >>> thread > >>> use patterns combined... And I've just had bright :) idea > >>> yesterday - > >>> it's also decent benchmark for whole threading system... I think it > >>> would be nice to test it with 10000 rounds, 4000 threads each (once > >>> cm3 > >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > >>> tests for > >>> mine. > >>> > >>> I think these extra data structures and global locks in PTHREAD are > >>> big > >>> efficiency killers. Benchmark will show. > >>> > >>> dd > >>> > >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >>>> I take all of your points seriously. One option would be to offer > >>>> your threads implementation as another build option for CM3. I'm > >>>> going to track down the bug I introduced recently and then we can > >>>> consider how to move forward. > >>>> > >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >>>> > >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>>>> > >>>>>>> Hello, > >>>>>>> > >>>>>>> Have been skimming through source of PTHREAD code... And I think > >>>>>>> job can > >>>>>>> be done without so much relying on how-they-did-it-before, esp > >>>>>>> with > >>>>>>> regard to list of waiters and similar internal and global > >>>>>>> structures. > >>>>>>> Also, I see number of global locks and I am sure they are > >>>>>>> congestion > >>>>>>> generators every now and while, esp in heavy threading > >>>>>>> situations. > >>>>>>> > >>>>>>> Of course, there is number of approaches to this multi-thread > >>>>>>> situations. Mine being one of very nonconservative use of > >>>>>>> threads, I > >>>>>>> think it is important to remain open to possibly very big > >>>>>>> number of > >>>>>>> threads running in single process - meaning scalability is > >>>>>>> one of > >>>>>>> primary objections... As global locks don't do well with > >>>>>>> scalability, I > >>>>>>> don't like "cm" and similar global congestion points. > >>>>>> > >>>>>> Yes, there are tensions between a thin/absent veneer between > >>>>>> language > >>>>>> threads and system threads. Most important are issues of > >>>>>> preserving > >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>>>> > >>>>> I know that paper, and being Modula-3 camp, I am - by definition - > >>>>> agreed to no-library-approach-for-threads :). > >>>>> > >>>>>> There are > >>>>>> also questions of portability and debuggability. > >>>>> > >>>>> Of course. That's why I am using only POSIX defined features, > >>>>> and > >>>>> when > >>>>> in doubt - ones used by Boehm in his famous GC :). > >>>>> > >>>>>> I agree that global > >>>>>> locks are to be avoided where they cause known contention, but > >>>>>> there > >>>>>> are tradeoffs there too. > >>>>> > >>>>> Global lock is bad, whatever reasons. > >>>>> > >>>>>> For large numbers of threads (as you appear > >>>>>> to need) I think we would need to adopt some other implementation > >>>>>> approach, possibly by multiplexing multiple lightweight user- > >>>>>> level > >>>>>> threads on some smaller number of heavyweight system-level > >>>>>> threads, > >>>>>> but then you run into scheduling and load-balancing problems. > >>>>> > >>>>> I've argued this before... With O(1) process scheduling > >>>>> available > >>>>> for > >>>>> four years from Linux, and in that time surely from everyone > >>>>> else... It > >>>>> would be bigger problem to maintain scheduling for special "BIG" > >>>>> cases > >>>>> inside our support libraries than to rely on operating system. > >>>>> It's > >>>>> good > >>>>> that mainstream OS people recognized threads as Need, and it is > >>>>> time now > >>>>> for us to accept they did it well - AT LAST. > >>>>> > >>>>> And, very important... I can't see what is heavyweight on system > >>>>> which > >>>>> does 10,000 context switches per second for 1500 threads with > >>>>> 2% CPU > >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads > >>>>> WERE > >>>>> heavyweight four years ago, and they are not, long since. Even > >>>>> Windows > >>>>> has lightweight kernel-space threads :). > >>>>> > >>>>>> > >>>>>>> We talked about this at least once before, and I think I > >>>>>>> remember > >>>>>>> you > >>>>>>> insisted on more compatibility than can be read from SPwM3. > >>>>>>> Do you > >>>>>>> think > >>>>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>>>> people to > >>>>>>> trash and test, and let everyone select what is best for their > >>>>>>> situation? > >>>>>> > >>>>>> What I wanted was a situation where programs would be able to run > >>>>>> with the same tools (e.g., showheap, showthread) under both user- > >>>>>> level and system threading. This goal has been achieved with the > >>>>>> current pthread-based implementation. > >>>>> > >>>>> It is good reasoning, and it's one of reasons I did not suggest > >>>>> replacement... I think mine version is less bloated and I know > >>>>> it's > >>>>> very > >>>>> good for long and stable process uptime we all expect from > >>>>> Modula-3 > >>>>> programs. But also, implied compatibilities outside of SPwM3 and > >>>>> direct > >>>>> demands from other parts of runtime were not on my list. These > >>>>> I've > >>>>> respected, and it looks like these are good production time > >>>>> criteria. As > >>>>> opposed to excellent development time criteria you based yours on. > >>>>> > >>>>>> Moreover, I wanted something > >>>>>> where variations in thread support from one system to another > >>>>>> could > >>>>>> be exploited most easily (such as for systems where thread > >>>>>> suspend/ > >>>>>> resume is provided as a primitive). Again, the current > >>>>>> implementation achieves this, and runs with minimal target- > >>>>>> specific > >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>>>> should be > >>>>>> relatively straightforward. > >>>>> > >>>>> I've not ported mine outside of LINUXLIBC6, but as it's > >>>>> extremmely > >>>>> POSIX, I see no problem. > >>>>> > >>>>>> > >>>>>>> Problems I had with my pthread implementation were all related > >>>>>>> to VM > >>>>>>> hell of earlier GC implementation... After you did that piece > >>>>>>> of art > >>>>>>> with new approach to GC, I expect infinite uptimes from my > >>>>>>> servers and > >>>>>>> bots :). Big thanks for that! > >>>>>> > >>>>>> Any native threading implementation is going to have problems > >>>>>> with > >>>>>> VM- > >>>>>> based memory management. I'm surprised to were able to get > >>>>>> anything > >>>>>> going with the VM-based GC. > >>>>> > >>>>> Anything is pretty much - I have heavy multithreaded servers > >>>>> running > >>>>> literally for years,,, one of them is up since January of 2004, it > >>>>> services few hundreds of connected users (and up to 1500 > >>>>> threads) at > >>>>> almost every moment and breaks only when system reboots :). All > >>>>> that > >>>>> with heavy integration of various C libraries. > >>>>> > >>>>>> > >>>>>>> > >>>>>>> dd > >>>>>>> -- > >>>>>>> Dragi?a Duri? > >>>>>> > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From darko at darko.org Tue May 22 15:13:09 2007 From: darko at darko.org (Darko) Date: Tue, 22 May 2007 15:13:09 +0200 Subject: [M3devel] Darwin Intel Data Alignment Message-ID: Hi, I'm having problems trying to use byte aligned fields in structures in the new Darwin Intel compiler. My understanding is that the processor accepts accesses on (word, etc) data not aligned on their normal boundaries. Is there something in the back end which means this is not possible? The <*LAZYALIGN*> pragma doesn't work properly and I get: "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** unaligned store_indirect type=11 s/a=32/16 "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable to find integer type? type=Int.32 s/o/a=32/64/16 when I compile. I think the platform independent alignment calculation code is correct, so I'm thinking this is a back end limitation. Any thoughts? Cheers, Darko. From hosking at cs.purdue.edu Tue May 22 17:22:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 22 May 2007 11:22:55 -0400 Subject: [M3devel] Darwin Intel Data Alignment In-Reply-To: References: Message-ID: Hmm. I am not familiar with this aspect of things. One thing would be to check if things work for you on LINUXLIBC6. Then we just need to figure out what is different between the Darwin and Linux Intel targets. Can you send me some test code that demonstrates the problem? On May 22, 2007, at 9:13 AM, Darko wrote: > Hi, > > I'm having problems trying to use byte aligned fields in structures > in the new Darwin Intel compiler. My understanding is that the > processor accepts accesses on (word, etc) data not aligned on their > normal boundaries. Is there something in the back end which means > this is not possible? > > The <*LAZYALIGN*> pragma doesn't work properly and I get: > > "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** > unaligned store_indirect type=11 s/a=32/16 > "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable > to find integer type? type=Int.32 s/o/a=32/64/16 > > when I compile. I think the platform independent alignment > calculation code is correct, so I'm thinking this is a back end > limitation. > > Any thoughts? > > > Cheers, > Darko. From rodney.bates at wichita.edu Sat May 26 23:48:17 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sat, 26 May 2007 16:48:17 -0500 Subject: [M3devel] Still can't bootstrap CM3 Message-ID: <4658AB21.5030100@wichita.edu> I still can't get a working cm3 built. I tried the following: Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there. Did 'cvs -z3 update -dAP' in the top of my copy of cm3. In scripts, did 'do-cm3-min.sh buildship', which ended with: ----------------------------------------------------------------------------- === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ --- building in LINUXLIBC6 --- ignoring ../src/m3overrides new source -> compiling ConvertPacking.m3 new "ConvertPacking.mo" -> archiving libm3.a Fatal Error: incomplete library ThreadPosix.i3: missing object types: _t1541f475 imported by: TimeStamp.m3 *** execution of failed *** ------------------------------------------------------------------------------ I believe this is the same symptom I had the last time I tried. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From rodney.bates at wichita.edu Sun May 27 18:23:51 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 27 May 2007 11:23:51 -0500 Subject: [M3devel] Can't load dynamically linked m3bundle Message-ID: <4659B097.5040908@wichita.edu> While trying to bootstrap CM3, I get this: ----------------------------------------------------------------------------------------------------- rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle -rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle rodney at runnymede:/usr/local/cm3/bin$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle -bash: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle /usr/bin/ldd: line 125: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue Mandriva Linux release 2007.0 (Cooker) for i586 Kernel 2.6.17-5mdv on an i686 / \l rodney at runnymede:/usr/local/cm3/bin$ ------------------------------------------------------------------------------------------------------ On an earlier Mandriva release, m3bundle turns out statically linked. In both cases, the m3makefile for m3bundle contains 'build_standalone ( )'. ------------------------------------------------------------------------------------------------------ [rodney at selkirk bin]$ pwd /usr/local/cm3/bin [rodney at selkirk bin]$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically linked, not stripped [rodney at selkirk bin]$ cat /etc/issue Mandrakelinux release 10.2 (Limited Edition 2005) for i586 Kernel 2.6.11-6mdk on an i686 / \l [rodney at selkirk bin]$ ------------------------------------------------------------------------------------------------------ I am guessing that there is some dynamic library that m3bundle wants that I don't have installed, but how can I find it? 'objdump -T m3bundle' gives a lot of entries, the most interesting ones naming GCC_3.0. The system where it fails has gcc 4.1.1 installed. I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a symbolic link to it, but to no avail. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Mon May 28 18:55:21 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 28 May 2007 12:55:21 -0400 Subject: [M3devel] Still can't bootstrap CM3 In-Reply-To: <4658AB21.5030100@wichita.edu> References: <4658AB21.5030100@wichita.edu> Message-ID: Did you do realclean before buildship? I have successfully bootstrapped from a clean CVS head using that cm3 binary on several Linux platforms. On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote: > I still can't get a working cm3 built. I tried the following: > > Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ > pub/hosking/m3/LINUXLIBC6/cm3.gz > and put it in /usr/local/cm3/bin, which is on my path and the > command 'cm3' finds it there. > > Did 'cvs -z3 update -dAP' in the top of my copy of cm3. > > In scripts, did 'do-cm3-min.sh buildship', which ended with: > > ---------------------------------------------------------------------- > ------- > === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === > +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && > cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > new source -> compiling ConvertPacking.m3 > new "ConvertPacking.mo" -> archiving libm3.a > > Fatal Error: incomplete library > > ThreadPosix.i3: missing object types: _t1541f475 > imported by: TimeStamp.m3 > *** execution of failed *** > > ---------------------------------------------------------------------- > -------- > > I believe this is the same symptom I had the last time I tried. > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu From ronny.forberger at elegosoft.com Wed May 30 15:24:16 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 30 May 2007 15:24:16 +0200 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap Message-ID: <465D7B00.8000205@elegosoft.com> Dear m3-Developers, I could not successfully build the cm3 compiler when invoking > ./do-cm3-std.sh buildship from the script directory as described in the documentation. The build process stops by the following: > === package /root/cm3/m3-sys/m3cgcat === > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > -> linking m3cgcat > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InnerLockMutex': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined > reference to `pthread_cond_wait' > [ some similar lines removed ] > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PushEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined > reference to `pthread_getspecific' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PopEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InitHandlers': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined > reference to `pthread_key_create' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined > reference to `pthread_setspecific' > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In > function `vfork': > (.text+0x1): undefined reference to `pthread_create' > collect2: ld returned 1 exit status > Fatal Error: package build failed > *** execution of failed *** Looks like gcc cannot link against the pthread-lib. I have already added > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] to the SYSTEM_LIBS array in cm3.cfg. This is a Debian 4.0 'etch' system (x86) where libpthread.a resides in /usr/lib. I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread arccordingly which become linked against libpthread.a properly. Do you have any glues? Best regards, Ronny -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From hosking at cs.purdue.edu Wed May 30 16:47:56 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 30 May 2007 10:47:56 -0400 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> References: <465D7B00.8000205@elegosoft.com> Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu> Can you use the -commands option to cm3 to show the link command? It certainly looks like it is not finding the pthread library. On May 30, 2007, at 9:24 AM, Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when invoking > > >> ./do-cm3-std.sh buildship > > > from the script directory as described in the documentation. > > The build process stops by the following: > >> === package /root/cm3/m3-sys/m3cgcat === >> +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/ >> cm3' +++ >> --- building in LINUXLIBC6 --- >> >> ignoring ../src/m3overrides >> >> -> linking m3cgcat >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: >> undefined reference to `pthread_cond_wait' >> > [ some similar lines removed ] > >> >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PushEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: >> undefined reference to `pthread_getspecific' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PopEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: >> undefined reference to `pthread_key_create' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: >> undefined reference to `pthread_setspecific' >> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): >> In function `vfork': >> (.text+0x1): undefined reference to `pthread_create' >> collect2: ld returned 1 exit status >> Fatal Error: package build failed >> *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > >> "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where libpthread.a > resides in /usr/lib. > > I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread > arccordingly which become linked against libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH ronny.forberger(at) > elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin > http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) > elego.de . From dabenavidesd at yahoo.es Thu May 31 01:42:04 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 31 May 2007 01:42:04 +0200 (CEST) Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com> Hi, Looks that you haven't used the SYSTEM_LIBORDER array, must be "PTHREAD" there in order to really link it. Maybe this little guide can help. Can have several omissions, but It works well in (K)ubuntu 6.06 and 6.10 http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000 --- Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when > invoking > > > > > ./do-cm3-std.sh buildship > > > from the script directory as described in the > documentation. > > The build process stops by the following: > > > === package /root/cm3/m3-sys/m3cgcat === > > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship > -DROOT='/root/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > -> linking m3cgcat > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InnerLockMutex': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: > undefined > > reference to `pthread_cond_wait' > > > [ some similar lines removed ] > > > > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PushEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: > undefined > > reference to `pthread_getspecific' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PopEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InitHandlers': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: > undefined > > reference to `pthread_key_create' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: > undefined > > reference to `pthread_setspecific' > > > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): > In > > function `vfork': > > (.text+0x1): undefined reference to > `pthread_create' > > collect2: ld returned 1 exit status > > Fatal Error: package build failed > > *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > > > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where > libpthread.a resides in > /usr/lib. > > I can build trivial C programs like gcc foo.c -L > /usr/lib -lpthread > arccordingly which become linked against > libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH > ronny.forberger(at)elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen > an admins(at)elego.de . > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From dragisha at m3w.org Wed May 2 07:45:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 07:45:52 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: <1178084752.8631.3.camel@faramir.m3w.org> Is it possible to get this fix separatelly from threading fixes? It's more important, esp since only program that crashes threading is TestThreads :) TIA, dd On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: > Yes, this fix is in CVS head. > > > On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > > > It looks like this fix never came through... With cvs-head CM3, > > unpickling of TextRefTbl.T instance still does not work. > > > > dd > > > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > >> Rodney, > >> > >> I've checked in fixes to RTTypeMap and clients that should restore > >> functional pickling support. For some reason my commit messages are > >> awaiting moderator approval (it seems my commit id is not the same as > >> my subscription to the m3commit list) before they can be sent out. > >> Please make sure that unpickling now works for you. > >> > >> Best regards, > >> > >> Tony > >> > >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > >> > >>> I tracked down a bug unpickling to the following. When executing > >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > >>> > >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > >>> ref^ := v.readRef(nelem.refType); > >>> END; > >>> > >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > >>> CheckStoreTraced expects ref to be a normal reference to an object, > >>> computes the address of its header (4 bytes less), and sets the > >>> dirty bit. > >>> > >>> But ConvertPacking is reading in to a field of a recently created > >>> object that is not first in the object. It computes ref to point > >>> to the field, and CheckStoreTraced actually steps on the previous > >>> real field instead of the header. > >>> > >>> It looks like the actions of CheckStoreTraced are needed, but I > >>> don't right off hand see how to recode Convert to fix this. It's > >>> use of ref can't be moved. Presumably, there could be other unsafe > >>> code elsewhere using the same technique to store data. > >>> > >>> Does the compiler need a different criterion on when to generate > >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > >>> still is pointing inside a traced object. Maybe people who modify > >>> objects in this way need to explicitly call CheckStoreTraced? > >>> > >>> -- > >>> ------------------------------------------------------------- > >>> Rodney M. Bates, retired assistant professor > >>> Dept. of Computer Science, Wichita State University > >>> Wichita, KS 67260-0083 > >>> 316-978-3922 > >>> rodney.bates at wichita.edu > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> Antony Hosking | Associate Professor > >> Dept of Computer Science | Office: +1 765 494-6001 > >> Purdue University | Mobile: +1 765 427-5484 > >> 250 N. University Street | Email: hosking at cs.purdue.edu > >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > >> _--_|\ > >> / \ > >> \_.--._/ ) > >> v / > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Wed May 2 08:18:22 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 2 May 2007 02:18:22 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> <1178084752.8631.3.camel@faramir.m3w.org> Message-ID: Just grab CVS from before 2007/03/2007 (when I broke GC). The pertinent fix was to RTTypeMap.m3, as at the current CVS head, which is dated 2007/01/09. Thread/GC-related fixes are coming -- I know what the problem is but am in the middle of refactoring the heap allocation code. On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote: > Is it possible to get this fix separatelly from threading fixes? > It's > more important, esp since only program that crashes threading is > TestThreads :) > > TIA, > dd > > On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: >> Yes, this fix is in CVS head. >> >> >> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: >> >>> It looks like this fix never came through... With cvs-head CM3, >>> unpickling of TextRefTbl.T instance still does not work. >>> >>> dd >>> >>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >>>> Rodney, >>>> >>>> I've checked in fixes to RTTypeMap and clients that should restore >>>> functional pickling support. For some reason my commit messages >>>> are >>>> awaiting moderator approval (it seems my commit id is not the >>>> same as >>>> my subscription to the m3commit list) before they can be sent out. >>>> Please make sure that unpickling now works for you. >>>> >>>> Best regards, >>>> >>>> Tony >>>> >>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >>>> >>>>> I tracked down a bug unpickling to the following. When executing >>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>>>> >>>>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>>>> ref^ := v.readRef(nelem.refType); >>>>> END; >>>>> >>>>> has a compiler-generated call on RTCollector.CheckStoreTraced >>>>> (ref). >>>>> CheckStoreTraced expects ref to be a normal reference to an >>>>> object, >>>>> computes the address of its header (4 bytes less), and sets the >>>>> dirty bit. >>>>> >>>>> But ConvertPacking is reading in to a field of a recently created >>>>> object that is not first in the object. It computes ref to point >>>>> to the field, and CheckStoreTraced actually steps on the previous >>>>> real field instead of the header. >>>>> >>>>> It looks like the actions of CheckStoreTraced are needed, but I >>>>> don't right off hand see how to recode Convert to fix this. It's >>>>> use of ref can't be moved. Presumably, there could be other >>>>> unsafe >>>>> code elsewhere using the same technique to store data. >>>>> >>>>> Does the compiler need a different criterion on when to generate >>>>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>>>> still is pointing inside a traced object. Maybe people who >>>>> modify >>>>> objects in this way need to explicitly call CheckStoreTraced? >>>>> >>>>> -- >>>>> ------------------------------------------------------------- >>>>> Rodney M. Bates, retired assistant professor >>>>> Dept. of Computer Science, Wichita State University >>>>> Wichita, KS 67260-0083 >>>>> 316-978-3922 >>>>> rodney.bates at wichita.edu >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Science | Office: +1 765 494-6001 >>>> Purdue University | Mobile: +1 765 427-5484 >>>> 250 N. University Street | Email: hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From ronny.forberger at elegosoft.com Wed May 2 11:24:14 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 02 May 2007 11:24:14 +0200 Subject: [M3devel] Just a test - please ignore Message-ID: <463858BE.1040000@elegosoft.com> -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From dragisha at m3w.org Wed May 2 23:11:42 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 23:11:42 +0200 Subject: [M3devel] formals initialization... NIL? Message-ID: <1178140302.8631.26.camel@faramir.m3w.org> 99 PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args: ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User } = 100 BEGIN 101 IF args[FIRST(args)] = NIL THEN 102 RAISE User( 103 NEW(RefSeq.T).fromArray( 104 ARRAY OF REFANY { e.getRef(f), e.getQName()} 105 ) (m3gdb) p args $1 = [16_00000001] Is this normal? 16_00000001 is NIL? Why not printed as such? -- Dragi?a Duri? From dragisha at m3w.org Sat May 5 09:44:07 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 05 May 2007 09:44:07 +0200 Subject: [M3devel] status of -linkall? Message-ID: <1178351048.21091.3.camel@faramir.m3w.org> I have a situation where -linkall works "as advertised", at least I don't see anything missing yet :). Same set of modules, when included in build of another program, fails to initialize... If I've let anyone forget it :), it's plugin architecture where modules plug to appropriate "connection points" simply by inclusion in m3makefile. Is it known problem with -linkall? dd -- Dragi?a Duri? From dragisha at m3w.org Mon May 7 08:03:45 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 07 May 2007 08:03:45 +0200 Subject: [M3devel] import/exports loops... dynamic loading of Modula-3 modules Message-ID: <1178517825.3925.23.camel@faramir.m3w.org> I am thinking about writing some loop detecting tool for (in)famous order-of-initialization sindrome. Anyone did it already? I think we really need something to help sort out of EXPORTS nightmare and it's influence on RTLinker behaviour. Also in news :), after I've tried to solve my problems with -linkall (and got very big executabales as consequence, together with almost working programs - at least until I try to make another similar build :) - dynamic loading of modules was logical next step. I've done it, and can confirm RTLinker.AddUnit works as advertised, any time one calls it :) It also solves my plugin architecture problem, as my plugin modules are not imported directly and thus their initialization is fully performed with adequate RTLinker.AddUnit call. If there's interest in how-to-make-dynamic-modules-happen, I can post it somewhere. My google search did not show ay Modula-3 work on that. dd -- Dragi?a Duri? From darko at darko.org Wed May 9 01:55:18 2007 From: darko at darko.org (Darko) Date: Wed, 9 May 2007 01:55:18 +0200 Subject: [M3devel] m3.org expiry Message-ID: It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. From wagner at elegosoft.com Wed May 9 07:57:41 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 9 May 2007 07:57:41 +0200 Subject: [M3devel] [darko@darko.org: m3.org expiry] Message-ID: <20070509055740.GA20162@elegosoft.com> Hi Farshad, I seem to remember that you tried to register m3.org for a period longer than a year, but now it's gone again: % host www.m3.org www.m3.org is an alias for resalehost.networksolutions.com. resalehost.networksolutions.com has address 205.178.189.128 What's the current state of things from your side? Should anybody else get and host m3.org? Olaf ----- Forwarded message from Darko ----- X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com To: Modula-3 developers From: Darko Date: Wed, 9 May 2007 01:55:18 +0200 X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed version=3.0.3 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Subject: [M3devel] m3.org expiry X-BeenThere: m3devel at elegosoft.com X-Mailman-Version: 2.1.9 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed May 16 17:21:12 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:21:12 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: TestThreads is still locking up with 3000 threads (it works with 2000 on my I386_DARWIN box). I am now tracking this down. On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed May 16 17:19:54 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:19:54 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> <1177281325.737.55.camel@faramir.m3w.org> <1177308751.737.63.camel@faramir.m3w.org> Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu> On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote: > Thank you. I am using your posted compiler for now. > > dd > > On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote: >> Something appears to be very broken with my most recent fixes for >> your threading problems (since Apr 19, when I put the last cm3.gz >> compiler bootstrap up). I will try to track this down tomorrow. I have tracked down and fixed this brokenness. CVS head should now behave reasonably. >> >> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote: >> >>> Any pointer on how to do this? >>> >>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am >>> using. >>> >>> dd >>> >>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote: >>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt >>>> for this fix to take effect. >>>> >>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: >>>> >>>>> This works just fine for me on the most recent CVS head for >>>>> I386_DARWIN. >>>>> >>>>> I wonder if you have not updated your compiler to match? There >>>>> was >>>>> a matching fix to the compiler. You should build and ship: >>>>> >>>>> m3gc-simple >>>>> m3core >>>>> libm3 >>>>> m3middle >>>>> m3linker >>>>> m3front >>>>> m3quake >>>>> cm3 >>>>> >>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your >>>>> cm3.cfg). This will get you a new compiler. >>>>> >>>>> Then, rebuild your CM3 installation from the CVS head. >>>>> >>>>> Begin forwarded message: >>>>> >>>>>> From: Dragi?a Duri? >>>>>> Date: April 22, 2007 5:25:33 PM EDT >>>>>> To: Tony Hosking >>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >>>>>> >>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>>>>>> Can you send me a stack dump from gdb at the point of the error? >>>>>>> Even better, do you have a simple example program I can use to >>>>>>> test >>>>>>> with? >>>>>> >>>>>> Here is one. Quick and dirty, but it shows problem. >>>>>> >>>>>> -- >>>>>> Dragi?a Duri? >>>>>> >>>>> >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Fri May 18 13:50:24 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 18 May 2007 13:50:24 +0200 Subject: [M3devel] Pickle compatibility... Message-ID: <1179489025.6698.3.camel@faramir.m3w.org> I have some data pickled by pm3 pickler... Now, reading these fails... First app I've tried had online quiz high scores pickled, but now I have to convert billing app :). What can I do to convert this? dd -- Dragi?a Duri? From hosking at cs.purdue.edu Sat May 19 22:00:36 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 19 May 2007 16:00:36 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> I just checked in a patch that is closer to your code, and that lets me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread creation fails for that many). On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Sat May 19 22:16:47 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 19 May 2007 22:16:47 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> Message-ID: <1179605808.8468.4.camel@faramir.m3w.org> I am running 4000, but it's not most important anyway... Did you see that (Un)LockMutex semantics I've enabled so one can overload methods invoked by LOCK .. DO? dd On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote: > I just checked in a patch that is closer to your code, and that lets > me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread > creation fails for that many). > > On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > > > After implementing that workaround for result code 11 in that > > SIGSUSPEND > > loop, every time during 1st or at most second pass (of 4000) it > > stucks. > > Not same place every time, though... I think there are RCs in > > LockHeap/SuspendAll logic. > > > > dd > > > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > >> Yes, good thinking. Tuning the threads systems is a good plan all > >> round. > >> > >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > >> > >>> Just a random thought... > >>> > >>> I don't think my TestThreads is something special, but it's few > >>> thread > >>> use patterns combined... And I've just had bright :) idea > >>> yesterday - > >>> it's also decent benchmark for whole threading system... I think it > >>> would be nice to test it with 10000 rounds, 4000 threads each (once > >>> cm3 > >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > >>> tests for > >>> mine. > >>> > >>> I think these extra data structures and global locks in PTHREAD are > >>> big > >>> efficiency killers. Benchmark will show. > >>> > >>> dd > >>> > >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >>>> I take all of your points seriously. One option would be to offer > >>>> your threads implementation as another build option for CM3. I'm > >>>> going to track down the bug I introduced recently and then we can > >>>> consider how to move forward. > >>>> > >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >>>> > >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>>>> > >>>>>>> Hello, > >>>>>>> > >>>>>>> Have been skimming through source of PTHREAD code... And I think > >>>>>>> job can > >>>>>>> be done without so much relying on how-they-did-it-before, esp > >>>>>>> with > >>>>>>> regard to list of waiters and similar internal and global > >>>>>>> structures. > >>>>>>> Also, I see number of global locks and I am sure they are > >>>>>>> congestion > >>>>>>> generators every now and while, esp in heavy threading > >>>>>>> situations. > >>>>>>> > >>>>>>> Of course, there is number of approaches to this multi-thread > >>>>>>> situations. Mine being one of very nonconservative use of > >>>>>>> threads, I > >>>>>>> think it is important to remain open to possibly very big > >>>>>>> number of > >>>>>>> threads running in single process - meaning scalability is > >>>>>>> one of > >>>>>>> primary objections... As global locks don't do well with > >>>>>>> scalability, I > >>>>>>> don't like "cm" and similar global congestion points. > >>>>>> > >>>>>> Yes, there are tensions between a thin/absent veneer between > >>>>>> language > >>>>>> threads and system threads. Most important are issues of > >>>>>> preserving > >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>>>> > >>>>> I know that paper, and being Modula-3 camp, I am - by definition - > >>>>> agreed to no-library-approach-for-threads :). > >>>>> > >>>>>> There are > >>>>>> also questions of portability and debuggability. > >>>>> > >>>>> Of course. That's why I am using only POSIX defined features, > >>>>> and > >>>>> when > >>>>> in doubt - ones used by Boehm in his famous GC :). > >>>>> > >>>>>> I agree that global > >>>>>> locks are to be avoided where they cause known contention, but > >>>>>> there > >>>>>> are tradeoffs there too. > >>>>> > >>>>> Global lock is bad, whatever reasons. > >>>>> > >>>>>> For large numbers of threads (as you appear > >>>>>> to need) I think we would need to adopt some other implementation > >>>>>> approach, possibly by multiplexing multiple lightweight user- > >>>>>> level > >>>>>> threads on some smaller number of heavyweight system-level > >>>>>> threads, > >>>>>> but then you run into scheduling and load-balancing problems. > >>>>> > >>>>> I've argued this before... With O(1) process scheduling > >>>>> available > >>>>> for > >>>>> four years from Linux, and in that time surely from everyone > >>>>> else... It > >>>>> would be bigger problem to maintain scheduling for special "BIG" > >>>>> cases > >>>>> inside our support libraries than to rely on operating system. > >>>>> It's > >>>>> good > >>>>> that mainstream OS people recognized threads as Need, and it is > >>>>> time now > >>>>> for us to accept they did it well - AT LAST. > >>>>> > >>>>> And, very important... I can't see what is heavyweight on system > >>>>> which > >>>>> does 10,000 context switches per second for 1500 threads with > >>>>> 2% CPU > >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads > >>>>> WERE > >>>>> heavyweight four years ago, and they are not, long since. Even > >>>>> Windows > >>>>> has lightweight kernel-space threads :). > >>>>> > >>>>>> > >>>>>>> We talked about this at least once before, and I think I > >>>>>>> remember > >>>>>>> you > >>>>>>> insisted on more compatibility than can be read from SPwM3. > >>>>>>> Do you > >>>>>>> think > >>>>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>>>> people to > >>>>>>> trash and test, and let everyone select what is best for their > >>>>>>> situation? > >>>>>> > >>>>>> What I wanted was a situation where programs would be able to run > >>>>>> with the same tools (e.g., showheap, showthread) under both user- > >>>>>> level and system threading. This goal has been achieved with the > >>>>>> current pthread-based implementation. > >>>>> > >>>>> It is good reasoning, and it's one of reasons I did not suggest > >>>>> replacement... I think mine version is less bloated and I know > >>>>> it's > >>>>> very > >>>>> good for long and stable process uptime we all expect from > >>>>> Modula-3 > >>>>> programs. But also, implied compatibilities outside of SPwM3 and > >>>>> direct > >>>>> demands from other parts of runtime were not on my list. These > >>>>> I've > >>>>> respected, and it looks like these are good production time > >>>>> criteria. As > >>>>> opposed to excellent development time criteria you based yours on. > >>>>> > >>>>>> Moreover, I wanted something > >>>>>> where variations in thread support from one system to another > >>>>>> could > >>>>>> be exploited most easily (such as for systems where thread > >>>>>> suspend/ > >>>>>> resume is provided as a primitive). Again, the current > >>>>>> implementation achieves this, and runs with minimal target- > >>>>>> specific > >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>>>> should be > >>>>>> relatively straightforward. > >>>>> > >>>>> I've not ported mine outside of LINUXLIBC6, but as it's > >>>>> extremmely > >>>>> POSIX, I see no problem. > >>>>> > >>>>>> > >>>>>>> Problems I had with my pthread implementation were all related > >>>>>>> to VM > >>>>>>> hell of earlier GC implementation... After you did that piece > >>>>>>> of art > >>>>>>> with new approach to GC, I expect infinite uptimes from my > >>>>>>> servers and > >>>>>>> bots :). Big thanks for that! > >>>>>> > >>>>>> Any native threading implementation is going to have problems > >>>>>> with > >>>>>> VM- > >>>>>> based memory management. I'm surprised to were able to get > >>>>>> anything > >>>>>> going with the VM-based GC. > >>>>> > >>>>> Anything is pretty much - I have heavy multithreaded servers > >>>>> running > >>>>> literally for years,,, one of them is up since January of 2004, it > >>>>> services few hundreds of connected users (and up to 1500 > >>>>> threads) at > >>>>> almost every moment and breaks only when system reboots :). All > >>>>> that > >>>>> with heavy integration of various C libraries. > >>>>> > >>>>>> > >>>>>>> > >>>>>>> dd > >>>>>>> -- > >>>>>>> Dragi?a Duri? > >>>>>> > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From darko at darko.org Tue May 22 15:13:09 2007 From: darko at darko.org (Darko) Date: Tue, 22 May 2007 15:13:09 +0200 Subject: [M3devel] Darwin Intel Data Alignment Message-ID: Hi, I'm having problems trying to use byte aligned fields in structures in the new Darwin Intel compiler. My understanding is that the processor accepts accesses on (word, etc) data not aligned on their normal boundaries. Is there something in the back end which means this is not possible? The <*LAZYALIGN*> pragma doesn't work properly and I get: "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** unaligned store_indirect type=11 s/a=32/16 "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable to find integer type? type=Int.32 s/o/a=32/64/16 when I compile. I think the platform independent alignment calculation code is correct, so I'm thinking this is a back end limitation. Any thoughts? Cheers, Darko. From hosking at cs.purdue.edu Tue May 22 17:22:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 22 May 2007 11:22:55 -0400 Subject: [M3devel] Darwin Intel Data Alignment In-Reply-To: References: Message-ID: Hmm. I am not familiar with this aspect of things. One thing would be to check if things work for you on LINUXLIBC6. Then we just need to figure out what is different between the Darwin and Linux Intel targets. Can you send me some test code that demonstrates the problem? On May 22, 2007, at 9:13 AM, Darko wrote: > Hi, > > I'm having problems trying to use byte aligned fields in structures > in the new Darwin Intel compiler. My understanding is that the > processor accepts accesses on (word, etc) data not aligned on their > normal boundaries. Is there something in the back end which means > this is not possible? > > The <*LAZYALIGN*> pragma doesn't work properly and I get: > > "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** > unaligned store_indirect type=11 s/a=32/16 > "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable > to find integer type? type=Int.32 s/o/a=32/64/16 > > when I compile. I think the platform independent alignment > calculation code is correct, so I'm thinking this is a back end > limitation. > > Any thoughts? > > > Cheers, > Darko. From rodney.bates at wichita.edu Sat May 26 23:48:17 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sat, 26 May 2007 16:48:17 -0500 Subject: [M3devel] Still can't bootstrap CM3 Message-ID: <4658AB21.5030100@wichita.edu> I still can't get a working cm3 built. I tried the following: Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there. Did 'cvs -z3 update -dAP' in the top of my copy of cm3. In scripts, did 'do-cm3-min.sh buildship', which ended with: ----------------------------------------------------------------------------- === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ --- building in LINUXLIBC6 --- ignoring ../src/m3overrides new source -> compiling ConvertPacking.m3 new "ConvertPacking.mo" -> archiving libm3.a Fatal Error: incomplete library ThreadPosix.i3: missing object types: _t1541f475 imported by: TimeStamp.m3 *** execution of failed *** ------------------------------------------------------------------------------ I believe this is the same symptom I had the last time I tried. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From rodney.bates at wichita.edu Sun May 27 18:23:51 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 27 May 2007 11:23:51 -0500 Subject: [M3devel] Can't load dynamically linked m3bundle Message-ID: <4659B097.5040908@wichita.edu> While trying to bootstrap CM3, I get this: ----------------------------------------------------------------------------------------------------- rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle -rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle rodney at runnymede:/usr/local/cm3/bin$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle -bash: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle /usr/bin/ldd: line 125: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue Mandriva Linux release 2007.0 (Cooker) for i586 Kernel 2.6.17-5mdv on an i686 / \l rodney at runnymede:/usr/local/cm3/bin$ ------------------------------------------------------------------------------------------------------ On an earlier Mandriva release, m3bundle turns out statically linked. In both cases, the m3makefile for m3bundle contains 'build_standalone ( )'. ------------------------------------------------------------------------------------------------------ [rodney at selkirk bin]$ pwd /usr/local/cm3/bin [rodney at selkirk bin]$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically linked, not stripped [rodney at selkirk bin]$ cat /etc/issue Mandrakelinux release 10.2 (Limited Edition 2005) for i586 Kernel 2.6.11-6mdk on an i686 / \l [rodney at selkirk bin]$ ------------------------------------------------------------------------------------------------------ I am guessing that there is some dynamic library that m3bundle wants that I don't have installed, but how can I find it? 'objdump -T m3bundle' gives a lot of entries, the most interesting ones naming GCC_3.0. The system where it fails has gcc 4.1.1 installed. I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a symbolic link to it, but to no avail. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Mon May 28 18:55:21 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 28 May 2007 12:55:21 -0400 Subject: [M3devel] Still can't bootstrap CM3 In-Reply-To: <4658AB21.5030100@wichita.edu> References: <4658AB21.5030100@wichita.edu> Message-ID: Did you do realclean before buildship? I have successfully bootstrapped from a clean CVS head using that cm3 binary on several Linux platforms. On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote: > I still can't get a working cm3 built. I tried the following: > > Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ > pub/hosking/m3/LINUXLIBC6/cm3.gz > and put it in /usr/local/cm3/bin, which is on my path and the > command 'cm3' finds it there. > > Did 'cvs -z3 update -dAP' in the top of my copy of cm3. > > In scripts, did 'do-cm3-min.sh buildship', which ended with: > > ---------------------------------------------------------------------- > ------- > === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === > +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && > cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > new source -> compiling ConvertPacking.m3 > new "ConvertPacking.mo" -> archiving libm3.a > > Fatal Error: incomplete library > > ThreadPosix.i3: missing object types: _t1541f475 > imported by: TimeStamp.m3 > *** execution of failed *** > > ---------------------------------------------------------------------- > -------- > > I believe this is the same symptom I had the last time I tried. > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu From ronny.forberger at elegosoft.com Wed May 30 15:24:16 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 30 May 2007 15:24:16 +0200 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap Message-ID: <465D7B00.8000205@elegosoft.com> Dear m3-Developers, I could not successfully build the cm3 compiler when invoking > ./do-cm3-std.sh buildship from the script directory as described in the documentation. The build process stops by the following: > === package /root/cm3/m3-sys/m3cgcat === > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > -> linking m3cgcat > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InnerLockMutex': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined > reference to `pthread_cond_wait' > [ some similar lines removed ] > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PushEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined > reference to `pthread_getspecific' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PopEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InitHandlers': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined > reference to `pthread_key_create' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined > reference to `pthread_setspecific' > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In > function `vfork': > (.text+0x1): undefined reference to `pthread_create' > collect2: ld returned 1 exit status > Fatal Error: package build failed > *** execution of failed *** Looks like gcc cannot link against the pthread-lib. I have already added > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] to the SYSTEM_LIBS array in cm3.cfg. This is a Debian 4.0 'etch' system (x86) where libpthread.a resides in /usr/lib. I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread arccordingly which become linked against libpthread.a properly. Do you have any glues? Best regards, Ronny -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From hosking at cs.purdue.edu Wed May 30 16:47:56 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 30 May 2007 10:47:56 -0400 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> References: <465D7B00.8000205@elegosoft.com> Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu> Can you use the -commands option to cm3 to show the link command? It certainly looks like it is not finding the pthread library. On May 30, 2007, at 9:24 AM, Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when invoking > > >> ./do-cm3-std.sh buildship > > > from the script directory as described in the documentation. > > The build process stops by the following: > >> === package /root/cm3/m3-sys/m3cgcat === >> +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/ >> cm3' +++ >> --- building in LINUXLIBC6 --- >> >> ignoring ../src/m3overrides >> >> -> linking m3cgcat >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: >> undefined reference to `pthread_cond_wait' >> > [ some similar lines removed ] > >> >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PushEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: >> undefined reference to `pthread_getspecific' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PopEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: >> undefined reference to `pthread_key_create' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: >> undefined reference to `pthread_setspecific' >> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): >> In function `vfork': >> (.text+0x1): undefined reference to `pthread_create' >> collect2: ld returned 1 exit status >> Fatal Error: package build failed >> *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > >> "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where libpthread.a > resides in /usr/lib. > > I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread > arccordingly which become linked against libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH ronny.forberger(at) > elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin > http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) > elego.de . From dabenavidesd at yahoo.es Thu May 31 01:42:04 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 31 May 2007 01:42:04 +0200 (CEST) Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com> Hi, Looks that you haven't used the SYSTEM_LIBORDER array, must be "PTHREAD" there in order to really link it. Maybe this little guide can help. Can have several omissions, but It works well in (K)ubuntu 6.06 and 6.10 http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000 --- Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when > invoking > > > > > ./do-cm3-std.sh buildship > > > from the script directory as described in the > documentation. > > The build process stops by the following: > > > === package /root/cm3/m3-sys/m3cgcat === > > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship > -DROOT='/root/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > -> linking m3cgcat > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InnerLockMutex': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: > undefined > > reference to `pthread_cond_wait' > > > [ some similar lines removed ] > > > > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PushEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: > undefined > > reference to `pthread_getspecific' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PopEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InitHandlers': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: > undefined > > reference to `pthread_key_create' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: > undefined > > reference to `pthread_setspecific' > > > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): > In > > function `vfork': > > (.text+0x1): undefined reference to > `pthread_create' > > collect2: ld returned 1 exit status > > Fatal Error: package build failed > > *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > > > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where > libpthread.a resides in > /usr/lib. > > I can build trivial C programs like gcc foo.c -L > /usr/lib -lpthread > arccordingly which become linked against > libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH > ronny.forberger(at)elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen > an admins(at)elego.de . > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From dragisha at m3w.org Wed May 2 07:45:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 07:45:52 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: <1178084752.8631.3.camel@faramir.m3w.org> Is it possible to get this fix separatelly from threading fixes? It's more important, esp since only program that crashes threading is TestThreads :) TIA, dd On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: > Yes, this fix is in CVS head. > > > On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > > > It looks like this fix never came through... With cvs-head CM3, > > unpickling of TextRefTbl.T instance still does not work. > > > > dd > > > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > >> Rodney, > >> > >> I've checked in fixes to RTTypeMap and clients that should restore > >> functional pickling support. For some reason my commit messages are > >> awaiting moderator approval (it seems my commit id is not the same as > >> my subscription to the m3commit list) before they can be sent out. > >> Please make sure that unpickling now works for you. > >> > >> Best regards, > >> > >> Tony > >> > >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > >> > >>> I tracked down a bug unpickling to the following. When executing > >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > >>> > >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > >>> ref^ := v.readRef(nelem.refType); > >>> END; > >>> > >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > >>> CheckStoreTraced expects ref to be a normal reference to an object, > >>> computes the address of its header (4 bytes less), and sets the > >>> dirty bit. > >>> > >>> But ConvertPacking is reading in to a field of a recently created > >>> object that is not first in the object. It computes ref to point > >>> to the field, and CheckStoreTraced actually steps on the previous > >>> real field instead of the header. > >>> > >>> It looks like the actions of CheckStoreTraced are needed, but I > >>> don't right off hand see how to recode Convert to fix this. It's > >>> use of ref can't be moved. Presumably, there could be other unsafe > >>> code elsewhere using the same technique to store data. > >>> > >>> Does the compiler need a different criterion on when to generate > >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > >>> still is pointing inside a traced object. Maybe people who modify > >>> objects in this way need to explicitly call CheckStoreTraced? > >>> > >>> -- > >>> ------------------------------------------------------------- > >>> Rodney M. Bates, retired assistant professor > >>> Dept. of Computer Science, Wichita State University > >>> Wichita, KS 67260-0083 > >>> 316-978-3922 > >>> rodney.bates at wichita.edu > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> Antony Hosking | Associate Professor > >> Dept of Computer Science | Office: +1 765 494-6001 > >> Purdue University | Mobile: +1 765 427-5484 > >> 250 N. University Street | Email: hosking at cs.purdue.edu > >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > >> _--_|\ > >> / \ > >> \_.--._/ ) > >> v / > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Wed May 2 08:18:22 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 2 May 2007 02:18:22 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1178084752.8631.3.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> <1178084752.8631.3.camel@faramir.m3w.org> Message-ID: Just grab CVS from before 2007/03/2007 (when I broke GC). The pertinent fix was to RTTypeMap.m3, as at the current CVS head, which is dated 2007/01/09. Thread/GC-related fixes are coming -- I know what the problem is but am in the middle of refactoring the heap allocation code. On May 2, 2007, at 1:45 AM, Dragi?a Duri? wrote: > Is it possible to get this fix separatelly from threading fixes? > It's > more important, esp since only program that crashes threading is > TestThreads :) > > TIA, > dd > > On Sun, 2007-04-22 at 15:22 -0400, Tony Hosking wrote: >> Yes, this fix is in CVS head. >> >> >> On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: >> >>> It looks like this fix never came through... With cvs-head CM3, >>> unpickling of TextRefTbl.T instance still does not work. >>> >>> dd >>> >>> On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >>>> Rodney, >>>> >>>> I've checked in fixes to RTTypeMap and clients that should restore >>>> functional pickling support. For some reason my commit messages >>>> are >>>> awaiting moderator approval (it seems my commit id is not the >>>> same as >>>> my subscription to the m3commit list) before they can be sent out. >>>> Please make sure that unpickling now works for you. >>>> >>>> Best regards, >>>> >>>> Tony >>>> >>>> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >>>> >>>>> I tracked down a bug unpickling to the following. When executing >>>>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>>>> >>>>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>>>> ref^ := v.readRef(nelem.refType); >>>>> END; >>>>> >>>>> has a compiler-generated call on RTCollector.CheckStoreTraced >>>>> (ref). >>>>> CheckStoreTraced expects ref to be a normal reference to an >>>>> object, >>>>> computes the address of its header (4 bytes less), and sets the >>>>> dirty bit. >>>>> >>>>> But ConvertPacking is reading in to a field of a recently created >>>>> object that is not first in the object. It computes ref to point >>>>> to the field, and CheckStoreTraced actually steps on the previous >>>>> real field instead of the header. >>>>> >>>>> It looks like the actions of CheckStoreTraced are needed, but I >>>>> don't right off hand see how to recode Convert to fix this. It's >>>>> use of ref can't be moved. Presumably, there could be other >>>>> unsafe >>>>> code elsewhere using the same technique to store data. >>>>> >>>>> Does the compiler need a different criterion on when to generate >>>>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>>>> still is pointing inside a traced object. Maybe people who >>>>> modify >>>>> objects in this way need to explicitly call CheckStoreTraced? >>>>> >>>>> -- >>>>> ------------------------------------------------------------- >>>>> Rodney M. Bates, retired assistant professor >>>>> Dept. of Computer Science, Wichita State University >>>>> Wichita, KS 67260-0083 >>>>> 316-978-3922 >>>>> rodney.bates at wichita.edu >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Science | Office: +1 765 494-6001 >>>> Purdue University | Mobile: +1 765 427-5484 >>>> 250 N. University Street | Email: hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From ronny.forberger at elegosoft.com Wed May 2 11:24:14 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 02 May 2007 11:24:14 +0200 Subject: [M3devel] Just a test - please ignore Message-ID: <463858BE.1040000@elegosoft.com> -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From dragisha at m3w.org Wed May 2 23:11:42 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 02 May 2007 23:11:42 +0200 Subject: [M3devel] formals initialization... NIL? Message-ID: <1178140302.8631.26.camel@faramir.m3w.org> 99 PROCEDURE Raise(e: XLReference.T; f: XLFrame.T; READONLY args: ARRAY OF REFANY := ARRAY OF REFANY {NIL}) RAISES { RuntimeError, User } = 100 BEGIN 101 IF args[FIRST(args)] = NIL THEN 102 RAISE User( 103 NEW(RefSeq.T).fromArray( 104 ARRAY OF REFANY { e.getRef(f), e.getQName()} 105 ) (m3gdb) p args $1 = [16_00000001] Is this normal? 16_00000001 is NIL? Why not printed as such? -- Dragi?a Duri? From dragisha at m3w.org Sat May 5 09:44:07 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 05 May 2007 09:44:07 +0200 Subject: [M3devel] status of -linkall? Message-ID: <1178351048.21091.3.camel@faramir.m3w.org> I have a situation where -linkall works "as advertised", at least I don't see anything missing yet :). Same set of modules, when included in build of another program, fails to initialize... If I've let anyone forget it :), it's plugin architecture where modules plug to appropriate "connection points" simply by inclusion in m3makefile. Is it known problem with -linkall? dd -- Dragi?a Duri? From dragisha at m3w.org Mon May 7 08:03:45 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 07 May 2007 08:03:45 +0200 Subject: [M3devel] import/exports loops... dynamic loading of Modula-3 modules Message-ID: <1178517825.3925.23.camel@faramir.m3w.org> I am thinking about writing some loop detecting tool for (in)famous order-of-initialization sindrome. Anyone did it already? I think we really need something to help sort out of EXPORTS nightmare and it's influence on RTLinker behaviour. Also in news :), after I've tried to solve my problems with -linkall (and got very big executabales as consequence, together with almost working programs - at least until I try to make another similar build :) - dynamic loading of modules was logical next step. I've done it, and can confirm RTLinker.AddUnit works as advertised, any time one calls it :) It also solves my plugin architecture problem, as my plugin modules are not imported directly and thus their initialization is fully performed with adequate RTLinker.AddUnit call. If there's interest in how-to-make-dynamic-modules-happen, I can post it somewhere. My google search did not show ay Modula-3 work on that. dd -- Dragi?a Duri? From darko at darko.org Wed May 9 01:55:18 2007 From: darko at darko.org (Darko) Date: Wed, 9 May 2007 01:55:18 +0200 Subject: [M3devel] m3.org expiry Message-ID: It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. From wagner at elegosoft.com Wed May 9 07:57:41 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 9 May 2007 07:57:41 +0200 Subject: [M3devel] [darko@darko.org: m3.org expiry] Message-ID: <20070509055740.GA20162@elegosoft.com> Hi Farshad, I seem to remember that you tried to register m3.org for a period longer than a year, but now it's gone again: % host www.m3.org www.m3.org is an alias for resalehost.networksolutions.com. resalehost.networksolutions.com has address 205.178.189.128 What's the current state of things from your side? Should anybody else get and host m3.org? Olaf ----- Forwarded message from Darko ----- X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com X-Virus-Scanned: Debian amavisd-new at mx0.elegosoft.com To: Modula-3 developers From: Darko Date: Wed, 9 May 2007 01:55:18 +0200 X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed version=3.0.3 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Subject: [M3devel] m3.org expiry X-BeenThere: m3devel at elegosoft.com X-Mailman-Version: 2.1.9 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , It seems its time for the annual m3.org domain expiry. Is anyone in touch with Farshad or whoever owns the domain? Darko. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed May 16 17:21:12 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:21:12 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: TestThreads is still locking up with 3000 threads (it works with 2000 on my I386_DARWIN box). I am now tracking this down. On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed May 16 17:19:54 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 16 May 2007 11:19:54 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177308751.737.63.camel@faramir.m3w.org> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> <1177281325.737.55.camel@faramir.m3w.org> <1177308751.737.63.camel@faramir.m3w.org> Message-ID: <3CFAC725-EAA5-480E-B8BB-B1A398CBEBB6@cs.purdue.edu> On Apr 23, 2007, at 2:12 AM, Dragi?a Duri? wrote: > Thank you. I am using your posted compiler for now. > > dd > > On Sun, 2007-04-22 at 23:03 -0400, Tony Hosking wrote: >> Something appears to be very broken with my most recent fixes for >> your threading problems (since Apr 19, when I put the last cm3.gz >> compiler bootstrap up). I will try to track this down tomorrow. I have tracked down and fixed this brokenness. CVS head should now behave reasonably. >> >> On Apr 22, 2007, at 6:35 PM, Dragi?a Duri? wrote: >> >>> Any pointer on how to do this? >>> >>> Or you maybe have rebuilt binary for LINUXLIBC6? Like this one I am >>> using. >>> >>> dd >>> >>> On Sun, 2007-04-22 at 18:01 -0400, Tony Hosking wrote: >>>> PS The bootstrap compilers on elegosoft probably may to be rebuilt >>>> for this fix to take effect. >>>> >>>> On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: >>>> >>>>> This works just fine for me on the most recent CVS head for >>>>> I386_DARWIN. >>>>> >>>>> I wonder if you have not updated your compiler to match? There >>>>> was >>>>> a matching fix to the compiler. You should build and ship: >>>>> >>>>> m3gc-simple >>>>> m3core >>>>> libm3 >>>>> m3middle >>>>> m3linker >>>>> m3front >>>>> m3quake >>>>> cm3 >>>>> >>>>> Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your >>>>> cm3.cfg). This will get you a new compiler. >>>>> >>>>> Then, rebuild your CM3 installation from the CVS head. >>>>> >>>>> Begin forwarded message: >>>>> >>>>>> From: Dragi?a Duri? >>>>>> Date: April 22, 2007 5:25:33 PM EDT >>>>>> To: Tony Hosking >>>>>> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >>>>>> >>>>>> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>>>>>> Can you send me a stack dump from gdb at the point of the error? >>>>>>> Even better, do you have a simple example program I can use to >>>>>>> test >>>>>>> with? >>>>>> >>>>>> Here is one. Quick and dirty, but it shows problem. >>>>>> >>>>>> -- >>>>>> Dragi?a Duri? >>>>>> >>>>> >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Fri May 18 13:50:24 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 18 May 2007 13:50:24 +0200 Subject: [M3devel] Pickle compatibility... Message-ID: <1179489025.6698.3.camel@faramir.m3w.org> I have some data pickled by pm3 pickler... Now, reading these fails... First app I've tried had online quiz high scores pickled, but now I have to convert billing app :). What can I do to convert this? dd -- Dragi?a Duri? From hosking at cs.purdue.edu Sat May 19 22:00:36 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 19 May 2007 16:00:36 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177518937.15060.6.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> Message-ID: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> I just checked in a patch that is closer to your code, and that lets me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread creation fails for that many). On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > After implementing that workaround for result code 11 in that > SIGSUSPEND > loop, every time during 1st or at most second pass (of 4000) it > stucks. > Not same place every time, though... I think there are RCs in > LockHeap/SuspendAll logic. > > dd > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: >> Yes, good thinking. Tuning the threads systems is a good plan all >> round. >> >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: >> >>> Just a random thought... >>> >>> I don't think my TestThreads is something special, but it's few >>> thread >>> use patterns combined... And I've just had bright :) idea >>> yesterday - >>> it's also decent benchmark for whole threading system... I think it >>> would be nice to test it with 10000 rounds, 4000 threads each (once >>> cm3 >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do >>> tests for >>> mine. >>> >>> I think these extra data structures and global locks in PTHREAD are >>> big >>> efficiency killers. Benchmark will show. >>> >>> dd >>> >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >>>> I take all of your points seriously. One option would be to offer >>>> your threads implementation as another build option for CM3. I'm >>>> going to track down the bug I introduced recently and then we can >>>> consider how to move forward. >>>> >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >>>> >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Have been skimming through source of PTHREAD code... And I think >>>>>>> job can >>>>>>> be done without so much relying on how-they-did-it-before, esp >>>>>>> with >>>>>>> regard to list of waiters and similar internal and global >>>>>>> structures. >>>>>>> Also, I see number of global locks and I am sure they are >>>>>>> congestion >>>>>>> generators every now and while, esp in heavy threading >>>>>>> situations. >>>>>>> >>>>>>> Of course, there is number of approaches to this multi-thread >>>>>>> situations. Mine being one of very nonconservative use of >>>>>>> threads, I >>>>>>> think it is important to remain open to possibly very big >>>>>>> number of >>>>>>> threads running in single process - meaning scalability is >>>>>>> one of >>>>>>> primary objections... As global locks don't do well with >>>>>>> scalability, I >>>>>>> don't like "cm" and similar global congestion points. >>>>>> >>>>>> Yes, there are tensions between a thin/absent veneer between >>>>>> language >>>>>> threads and system threads. Most important are issues of >>>>>> preserving >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>>>> >>>>> I know that paper, and being Modula-3 camp, I am - by definition - >>>>> agreed to no-library-approach-for-threads :). >>>>> >>>>>> There are >>>>>> also questions of portability and debuggability. >>>>> >>>>> Of course. That's why I am using only POSIX defined features, >>>>> and >>>>> when >>>>> in doubt - ones used by Boehm in his famous GC :). >>>>> >>>>>> I agree that global >>>>>> locks are to be avoided where they cause known contention, but >>>>>> there >>>>>> are tradeoffs there too. >>>>> >>>>> Global lock is bad, whatever reasons. >>>>> >>>>>> For large numbers of threads (as you appear >>>>>> to need) I think we would need to adopt some other implementation >>>>>> approach, possibly by multiplexing multiple lightweight user- >>>>>> level >>>>>> threads on some smaller number of heavyweight system-level >>>>>> threads, >>>>>> but then you run into scheduling and load-balancing problems. >>>>> >>>>> I've argued this before... With O(1) process scheduling >>>>> available >>>>> for >>>>> four years from Linux, and in that time surely from everyone >>>>> else... It >>>>> would be bigger problem to maintain scheduling for special "BIG" >>>>> cases >>>>> inside our support libraries than to rely on operating system. >>>>> It's >>>>> good >>>>> that mainstream OS people recognized threads as Need, and it is >>>>> time now >>>>> for us to accept they did it well - AT LAST. >>>>> >>>>> And, very important... I can't see what is heavyweight on system >>>>> which >>>>> does 10,000 context switches per second for 1500 threads with >>>>> 2% CPU >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads >>>>> WERE >>>>> heavyweight four years ago, and they are not, long since. Even >>>>> Windows >>>>> has lightweight kernel-space threads :). >>>>> >>>>>> >>>>>>> We talked about this at least once before, and I think I >>>>>>> remember >>>>>>> you >>>>>>> insisted on more compatibility than can be read from SPwM3. >>>>>>> Do you >>>>>>> think >>>>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>>>> people to >>>>>>> trash and test, and let everyone select what is best for their >>>>>>> situation? >>>>>> >>>>>> What I wanted was a situation where programs would be able to run >>>>>> with the same tools (e.g., showheap, showthread) under both user- >>>>>> level and system threading. This goal has been achieved with the >>>>>> current pthread-based implementation. >>>>> >>>>> It is good reasoning, and it's one of reasons I did not suggest >>>>> replacement... I think mine version is less bloated and I know >>>>> it's >>>>> very >>>>> good for long and stable process uptime we all expect from >>>>> Modula-3 >>>>> programs. But also, implied compatibilities outside of SPwM3 and >>>>> direct >>>>> demands from other parts of runtime were not on my list. These >>>>> I've >>>>> respected, and it looks like these are good production time >>>>> criteria. As >>>>> opposed to excellent development time criteria you based yours on. >>>>> >>>>>> Moreover, I wanted something >>>>>> where variations in thread support from one system to another >>>>>> could >>>>>> be exploited most easily (such as for systems where thread >>>>>> suspend/ >>>>>> resume is provided as a primitive). Again, the current >>>>>> implementation achieves this, and runs with minimal target- >>>>>> specific >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>>>> should be >>>>>> relatively straightforward. >>>>> >>>>> I've not ported mine outside of LINUXLIBC6, but as it's >>>>> extremmely >>>>> POSIX, I see no problem. >>>>> >>>>>> >>>>>>> Problems I had with my pthread implementation were all related >>>>>>> to VM >>>>>>> hell of earlier GC implementation... After you did that piece >>>>>>> of art >>>>>>> with new approach to GC, I expect infinite uptimes from my >>>>>>> servers and >>>>>>> bots :). Big thanks for that! >>>>>> >>>>>> Any native threading implementation is going to have problems >>>>>> with >>>>>> VM- >>>>>> based memory management. I'm surprised to were able to get >>>>>> anything >>>>>> going with the VM-based GC. >>>>> >>>>> Anything is pretty much - I have heavy multithreaded servers >>>>> running >>>>> literally for years,,, one of them is up since January of 2004, it >>>>> services few hundreds of connected users (and up to 1500 >>>>> threads) at >>>>> almost every moment and breaks only when system reboots :). All >>>>> that >>>>> with heavy integration of various C libraries. >>>>> >>>>>> >>>>>>> >>>>>>> dd >>>>>>> -- >>>>>>> Dragi?a Duri? >>>>>> >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From dragisha at m3w.org Sat May 19 22:16:47 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 19 May 2007 22:16:47 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> <1177518937.15060.6.camel@faramir.m3w.org> <0D338624-DA64-4BCA-ABC9-AF2AA5F8E618@cs.purdue.edu> Message-ID: <1179605808.8468.4.camel@faramir.m3w.org> I am running 4000, but it's not most important anyway... Did you see that (Un)LockMutex semantics I've enabled so one can overload methods invoked by LOCK .. DO? dd On Sat, 2007-05-19 at 16:00 -0400, Tony Hosking wrote: > I just checked in a patch that is closer to your code, and that lets > me run 3200 threads on LINUXLIBC6 (I can't do 3500 because thread > creation fails for that many). > > On Apr 25, 2007, at 12:35 PM, Dragi?a Duri? wrote: > > > After implementing that workaround for result code 11 in that > > SIGSUSPEND > > loop, every time during 1st or at most second pass (of 4000) it > > stucks. > > Not same place every time, though... I think there are RCs in > > LockHeap/SuspendAll logic. > > > > dd > > > > On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > >> Yes, good thinking. Tuning the threads systems is a good plan all > >> round. > >> > >> On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > >> > >>> Just a random thought... > >>> > >>> I don't think my TestThreads is something special, but it's few > >>> thread > >>> use patterns combined... And I've just had bright :) idea > >>> yesterday - > >>> it's also decent benchmark for whole threading system... I think it > >>> would be nice to test it with 10000 rounds, 4000 threads each (once > >>> cm3 > >>> cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > >>> tests for > >>> mine. > >>> > >>> I think these extra data structures and global locks in PTHREAD are > >>> big > >>> efficiency killers. Benchmark will show. > >>> > >>> dd > >>> > >>> On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >>>> I take all of your points seriously. One option would be to offer > >>>> your threads implementation as another build option for CM3. I'm > >>>> going to track down the bug I introduced recently and then we can > >>>> consider how to move forward. > >>>> > >>>> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >>>> > >>>>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>>>> > >>>>>>> Hello, > >>>>>>> > >>>>>>> Have been skimming through source of PTHREAD code... And I think > >>>>>>> job can > >>>>>>> be done without so much relying on how-they-did-it-before, esp > >>>>>>> with > >>>>>>> regard to list of waiters and similar internal and global > >>>>>>> structures. > >>>>>>> Also, I see number of global locks and I am sure they are > >>>>>>> congestion > >>>>>>> generators every now and while, esp in heavy threading > >>>>>>> situations. > >>>>>>> > >>>>>>> Of course, there is number of approaches to this multi-thread > >>>>>>> situations. Mine being one of very nonconservative use of > >>>>>>> threads, I > >>>>>>> think it is important to remain open to possibly very big > >>>>>>> number of > >>>>>>> threads running in single process - meaning scalability is > >>>>>>> one of > >>>>>>> primary objections... As global locks don't do well with > >>>>>>> scalability, I > >>>>>>> don't like "cm" and similar global congestion points. > >>>>>> > >>>>>> Yes, there are tensions between a thin/absent veneer between > >>>>>> language > >>>>>> threads and system threads. Most important are issues of > >>>>>> preserving > >>>>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>>>> > >>>>> I know that paper, and being Modula-3 camp, I am - by definition - > >>>>> agreed to no-library-approach-for-threads :). > >>>>> > >>>>>> There are > >>>>>> also questions of portability and debuggability. > >>>>> > >>>>> Of course. That's why I am using only POSIX defined features, > >>>>> and > >>>>> when > >>>>> in doubt - ones used by Boehm in his famous GC :). > >>>>> > >>>>>> I agree that global > >>>>>> locks are to be avoided where they cause known contention, but > >>>>>> there > >>>>>> are tradeoffs there too. > >>>>> > >>>>> Global lock is bad, whatever reasons. > >>>>> > >>>>>> For large numbers of threads (as you appear > >>>>>> to need) I think we would need to adopt some other implementation > >>>>>> approach, possibly by multiplexing multiple lightweight user- > >>>>>> level > >>>>>> threads on some smaller number of heavyweight system-level > >>>>>> threads, > >>>>>> but then you run into scheduling and load-balancing problems. > >>>>> > >>>>> I've argued this before... With O(1) process scheduling > >>>>> available > >>>>> for > >>>>> four years from Linux, and in that time surely from everyone > >>>>> else... It > >>>>> would be bigger problem to maintain scheduling for special "BIG" > >>>>> cases > >>>>> inside our support libraries than to rely on operating system. > >>>>> It's > >>>>> good > >>>>> that mainstream OS people recognized threads as Need, and it is > >>>>> time now > >>>>> for us to accept they did it well - AT LAST. > >>>>> > >>>>> And, very important... I can't see what is heavyweight on system > >>>>> which > >>>>> does 10,000 context switches per second for 1500 threads with > >>>>> 2% CPU > >>>>> load. And all this in 2004 on some 1.something GHz CPU. Threads > >>>>> WERE > >>>>> heavyweight four years ago, and they are not, long since. Even > >>>>> Windows > >>>>> has lightweight kernel-space threads :). > >>>>> > >>>>>> > >>>>>>> We talked about this at least once before, and I think I > >>>>>>> remember > >>>>>>> you > >>>>>>> insisted on more compatibility than can be read from SPwM3. > >>>>>>> Do you > >>>>>>> think > >>>>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>>>> people to > >>>>>>> trash and test, and let everyone select what is best for their > >>>>>>> situation? > >>>>>> > >>>>>> What I wanted was a situation where programs would be able to run > >>>>>> with the same tools (e.g., showheap, showthread) under both user- > >>>>>> level and system threading. This goal has been achieved with the > >>>>>> current pthread-based implementation. > >>>>> > >>>>> It is good reasoning, and it's one of reasons I did not suggest > >>>>> replacement... I think mine version is less bloated and I know > >>>>> it's > >>>>> very > >>>>> good for long and stable process uptime we all expect from > >>>>> Modula-3 > >>>>> programs. But also, implied compatibilities outside of SPwM3 and > >>>>> direct > >>>>> demands from other parts of runtime were not on my list. These > >>>>> I've > >>>>> respected, and it looks like these are good production time > >>>>> criteria. As > >>>>> opposed to excellent development time criteria you based yours on. > >>>>> > >>>>>> Moreover, I wanted something > >>>>>> where variations in thread support from one system to another > >>>>>> could > >>>>>> be exploited most easily (such as for systems where thread > >>>>>> suspend/ > >>>>>> resume is provided as a primitive). Again, the current > >>>>>> implementation achieves this, and runs with minimal target- > >>>>>> specific > >>>>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>>>> should be > >>>>>> relatively straightforward. > >>>>> > >>>>> I've not ported mine outside of LINUXLIBC6, but as it's > >>>>> extremmely > >>>>> POSIX, I see no problem. > >>>>> > >>>>>> > >>>>>>> Problems I had with my pthread implementation were all related > >>>>>>> to VM > >>>>>>> hell of earlier GC implementation... After you did that piece > >>>>>>> of art > >>>>>>> with new approach to GC, I expect infinite uptimes from my > >>>>>>> servers and > >>>>>>> bots :). Big thanks for that! > >>>>>> > >>>>>> Any native threading implementation is going to have problems > >>>>>> with > >>>>>> VM- > >>>>>> based memory management. I'm surprised to were able to get > >>>>>> anything > >>>>>> going with the VM-based GC. > >>>>> > >>>>> Anything is pretty much - I have heavy multithreaded servers > >>>>> running > >>>>> literally for years,,, one of them is up since January of 2004, it > >>>>> services few hundreds of connected users (and up to 1500 > >>>>> threads) at > >>>>> almost every moment and breaks only when system reboots :). All > >>>>> that > >>>>> with heavy integration of various C libraries. > >>>>> > >>>>>> > >>>>>>> > >>>>>>> dd > >>>>>>> -- > >>>>>>> Dragi?a Duri? > >>>>>> > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From darko at darko.org Tue May 22 15:13:09 2007 From: darko at darko.org (Darko) Date: Tue, 22 May 2007 15:13:09 +0200 Subject: [M3devel] Darwin Intel Data Alignment Message-ID: Hi, I'm having problems trying to use byte aligned fields in structures in the new Darwin Intel compiler. My understanding is that the processor accepts accesses on (word, etc) data not aligned on their normal boundaries. Is there something in the back end which means this is not possible? The <*LAZYALIGN*> pragma doesn't work properly and I get: "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** unaligned store_indirect type=11 s/a=32/16 "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable to find integer type? type=Int.32 s/o/a=32/64/16 when I compile. I think the platform independent alignment calculation code is correct, so I'm thinking this is a back end limitation. Any thoughts? Cheers, Darko. From hosking at cs.purdue.edu Tue May 22 17:22:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 22 May 2007 11:22:55 -0400 Subject: [M3devel] Darwin Intel Data Alignment In-Reply-To: References: Message-ID: Hmm. I am not familiar with this aspect of things. One thing would be to check if things work for you on LINUXLIBC6. Then we just need to figure out what is different between the Darwin and Linux Intel targets. Can you send me some test code that demonstrates the problem? On May 22, 2007, at 9:13 AM, Darko wrote: > Hi, > > I'm having problems trying to use byte aligned fields in structures > in the new Darwin Intel compiler. My understanding is that the > processor accepts accesses on (word, etc) data not aligned on their > normal boundaries. Is there something in the back end which means > this is not possible? > > The <*LAZYALIGN*> pragma doesn't work properly and I get: > > "../src/mac/PalPlat.m3", line 1525: ** INTERNAL CG ERROR *** > unaligned store_indirect type=11 s/a=32/16 > "../src/mac/PalPlat.m3", line 2977: ** INTERNAL CG ERROR *** unable > to find integer type? type=Int.32 s/o/a=32/64/16 > > when I compile. I think the platform independent alignment > calculation code is correct, so I'm thinking this is a back end > limitation. > > Any thoughts? > > > Cheers, > Darko. From rodney.bates at wichita.edu Sat May 26 23:48:17 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sat, 26 May 2007 16:48:17 -0500 Subject: [M3devel] Still can't bootstrap CM3 Message-ID: <4658AB21.5030100@wichita.edu> I still can't get a working cm3 built. I tried the following: Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz and put it in /usr/local/cm3/bin, which is on my path and the command 'cm3' finds it there. Did 'cvs -z3 update -dAP' in the top of my copy of cm3. In scripts, did 'do-cm3-min.sh buildship', which ended with: ----------------------------------------------------------------------------- === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ --- building in LINUXLIBC6 --- ignoring ../src/m3overrides new source -> compiling ConvertPacking.m3 new "ConvertPacking.mo" -> archiving libm3.a Fatal Error: incomplete library ThreadPosix.i3: missing object types: _t1541f475 imported by: TimeStamp.m3 *** execution of failed *** ------------------------------------------------------------------------------ I believe this is the same symptom I had the last time I tried. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From rodney.bates at wichita.edu Sun May 27 18:23:51 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 27 May 2007 11:23:51 -0500 Subject: [M3devel] Can't load dynamically linked m3bundle Message-ID: <4659B097.5040908@wichita.edu> While trying to bootstrap CM3, I get this: ----------------------------------------------------------------------------------------------------- rodney at runnymede:/usr/local/cm3/bin$ ls -l m3bundle -rwxr-xr-x 1 rodney rodney 1438195 May 26 17:11 m3bundle rodney at runnymede:/usr/local/cm3/bin$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped rodney at runnymede:/usr/local/cm3/bin$ ./m3bundle -bash: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ ldd m3bundle /usr/bin/ldd: line 125: ./m3bundle: No such file or directory rodney at runnymede:/usr/local/cm3/bin$ cat /etc/issue Mandriva Linux release 2007.0 (Cooker) for i586 Kernel 2.6.17-5mdv on an i686 / \l rodney at runnymede:/usr/local/cm3/bin$ ------------------------------------------------------------------------------------------------------ On an earlier Mandriva release, m3bundle turns out statically linked. In both cases, the m3makefile for m3bundle contains 'build_standalone ( )'. ------------------------------------------------------------------------------------------------------ [rodney at selkirk bin]$ pwd /usr/local/cm3/bin [rodney at selkirk bin]$ file m3bundle m3bundle: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, statically linked, not stripped [rodney at selkirk bin]$ cat /etc/issue Mandrakelinux release 10.2 (Limited Edition 2005) for i586 Kernel 2.6.11-6mdk on an i686 / \l [rodney at selkirk bin]$ ------------------------------------------------------------------------------------------------------ I am guessing that there is some dynamic library that m3bundle wants that I don't have installed, but how can I find it? 'objdump -T m3bundle' gives a lot of entries, the most interesting ones naming GCC_3.0. The system where it fails has gcc 4.1.1 installed. I tried temporarily putting in a copy of libgcc_s-3.4.3.so.1 and making libgcc_s.so.1 a symbolic link to it, but to no avail. -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Mon May 28 18:55:21 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 28 May 2007 12:55:21 -0400 Subject: [M3devel] Still can't bootstrap CM3 In-Reply-To: <4658AB21.5030100@wichita.edu> References: <4658AB21.5030100@wichita.edu> Message-ID: Did you do realclean before buildship? I have successfully bootstrapped from a clean CVS head using that cm3 binary on several Linux platforms. On May 26, 2007, at 5:48 PM, Rodney M. Bates wrote: > I still can't get a working cm3 built. I tried the following: > > Got the bootstrap compiler executable from ftp://ftp.cs.purdue.edu/ > pub/hosking/m3/LINUXLIBC6/cm3.gz > and put it in /usr/local/cm3/bin, which is on my path and the > command 'cm3' finds it there. > > Did 'cvs -z3 update -dAP' in the top of my copy of cm3. > > In scripts, did 'do-cm3-min.sh buildship', which ended with: > > ---------------------------------------------------------------------- > ------- > === package /home/rodney/proj/m3/cm3-fresh/cm3/m3-libs/libm3 === > +++ cm3 -build -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' && > cm3 -ship -DROOT='/home/rodney/proj/m3/cm3-fresh/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > new source -> compiling ConvertPacking.m3 > new "ConvertPacking.mo" -> archiving libm3.a > > Fatal Error: incomplete library > > ThreadPosix.i3: missing object types: _t1541f475 > imported by: TimeStamp.m3 > *** execution of failed *** > > ---------------------------------------------------------------------- > -------- > > I believe this is the same symptom I had the last time I tried. > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu From ronny.forberger at elegosoft.com Wed May 30 15:24:16 2007 From: ronny.forberger at elegosoft.com (Ronny Forberger) Date: Wed, 30 May 2007 15:24:16 +0200 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap Message-ID: <465D7B00.8000205@elegosoft.com> Dear m3-Developers, I could not successfully build the cm3 compiler when invoking > ./do-cm3-std.sh buildship from the script directory as described in the documentation. The build process stops by the following: > === package /root/cm3/m3-sys/m3cgcat === > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/cm3' +++ > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > -> linking m3cgcat > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InnerLockMutex': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: undefined > reference to `pthread_cond_wait' > [ some similar lines removed ] > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PushEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: undefined > reference to `pthread_getspecific' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `RTHooks__PopEFrame': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: undefined > reference to `pthread_setspecific' > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): In > function `ThreadPThread__InitHandlers': > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: undefined > reference to `pthread_key_create' > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: undefined > reference to `pthread_setspecific' > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): In > function `vfork': > (.text+0x1): undefined reference to `pthread_create' > collect2: ld returned 1 exit status > Fatal Error: package build failed > *** execution of failed *** Looks like gcc cannot link against the pthread-lib. I have already added > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] to the SYSTEM_LIBS array in cm3.cfg. This is a Debian 4.0 'etch' system (x86) where libpthread.a resides in /usr/lib. I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread arccordingly which become linked against libpthread.a properly. Do you have any glues? Best regards, Ronny -- Ronny Forberger Systemadministrator elego Software Solutions GmbH ronny.forberger(at)elegosoft.com Ohmstra?e 9, 10179 Berlin HRB 77719 Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner Bitte senden Sie Ihre technischen Support-Anfragen an admins(at)elego.de . From hosking at cs.purdue.edu Wed May 30 16:47:56 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 30 May 2007 10:47:56 -0400 Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> References: <465D7B00.8000205@elegosoft.com> Message-ID: <55F02DB8-0B5F-48F8-B821-1295EB3A0DD7@cs.purdue.edu> Can you use the -commands option to cm3 to show the link command? It certainly looks like it is not finding the pthread library. On May 30, 2007, at 9:24 AM, Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when invoking > > >> ./do-cm3-std.sh buildship > > > from the script directory as described in the documentation. > > The build process stops by the following: > >> === package /root/cm3/m3-sys/m3cgcat === >> +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship -DROOT='/root/ >> cm3' +++ >> --- building in LINUXLIBC6 --- >> >> ignoring ../src/m3overrides >> >> -> linking m3cgcat >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InnerLockMutex': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: >> undefined reference to `pthread_cond_wait' >> > [ some similar lines removed ] > >> >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PushEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: >> undefined reference to `pthread_getspecific' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `RTHooks__PopEFrame': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: >> undefined reference to `pthread_setspecific' >> /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a >> (ThreadPThread.mo): In function `ThreadPThread__InitHandlers': >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: >> undefined reference to `pthread_key_create' >> /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: >> undefined reference to `pthread_setspecific' >> /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): >> In function `vfork': >> (.text+0x1): undefined reference to `pthread_create' >> collect2: ld returned 1 exit status >> Fatal Error: package build failed >> *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > >> "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where libpthread.a > resides in /usr/lib. > > I can build trivial C programs like gcc foo.c -L /usr/lib -lpthread > arccordingly which become linked against libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH ronny.forberger(at) > elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der Gesellschaft: Berlin > http://www.elegosoft.com Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen an admins(at) > elego.de . From dabenavidesd at yahoo.es Thu May 31 01:42:04 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 31 May 2007 01:42:04 +0200 (CEST) Subject: [M3devel] [cm3] unresolved pthread symbols during bootstrap In-Reply-To: <465D7B00.8000205@elegosoft.com> Message-ID: <20070530234204.77087.qmail@web27115.mail.ukl.yahoo.com> Hi, Looks that you haven't used the SYSTEM_LIBORDER array, must be "PTHREAD" there in order to really link it. Maybe this little guide can help. Can have several omissions, but It works well in (K)ubuntu 6.06 and 6.10 http://es.geocities.com/dabenavidesd/InstallationGuidecm3-Modula-3.html#SECTION00033000000000000000 --- Ronny Forberger wrote: > Dear m3-Developers, > > I could not successfully build the cm3 compiler when > invoking > > > > > ./do-cm3-std.sh buildship > > > from the script directory as described in the > documentation. > > The build process stops by the following: > > > === package /root/cm3/m3-sys/m3cgcat === > > +++ cm3 -build -DROOT='/root/cm3' && cm3 -ship > -DROOT='/root/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > -> linking m3cgcat > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InnerLockMutex': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:142: > undefined > > reference to `pthread_cond_wait' > > > [ some similar lines removed ] > > > > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PushEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1429: > undefined > > reference to `pthread_getspecific' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1430: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `RTHooks__PopEFrame': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1437: > undefined > > reference to `pthread_setspecific' > > > /usr/local/cm3/pkg/m3core/LINUXLIBC6/libm3core.a(ThreadPThread.mo): > In > > function `ThreadPThread__InitHandlers': > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1442: > undefined > > reference to `pthread_key_create' > > > /root/cm3/m3-libs/m3core/LINUXLIBC6/ThreadPThread.m3:1443: > undefined > > reference to `pthread_setspecific' > > > /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/libc.a(vfork.o): > In > > function `vfork': > > (.text+0x1): undefined reference to > `pthread_create' > > collect2: ld returned 1 exit status > > Fatal Error: package build failed > > *** execution of failed *** > > > Looks like gcc cannot link against the pthread-lib. > > I have already added > > > "PTHREAD" : [ "-L/usr/lib", "-lpthread" ] > > to the SYSTEM_LIBS array in cm3.cfg. > > This is a Debian 4.0 'etch' system (x86) where > libpthread.a resides in > /usr/lib. > > I can build trivial C programs like gcc foo.c -L > /usr/lib -lpthread > arccordingly which become linked against > libpthread.a properly. > > > Do you have any glues? > > Best regards, > > Ronny > > -- > Ronny Forberger > Systemadministrator > > elego Software Solutions GmbH > ronny.forberger(at)elegosoft.com > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > Bitte senden Sie Ihre technischen Support-Anfragen > an admins(at)elego.de . > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com