From darko at darko.org Tue Apr 3 15:33:23 2007 From: darko at darko.org (Darko) Date: Tue, 3 Apr 2007 15:33:23 +0200 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> Message-ID: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> I'm not sure if this has been resolved in some more proper way, but for the record, the warnings appearing after installation of the XCode 2.4.1 tools can be gotten rid of by changing the following in the config file: proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-o", object) end so it reads proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-W -o", object) end It's pretty obvious, but just in case anyone goes searching these archives for an answer... On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: > >> >> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >> >>> For some reason your .stabs entries are different than mine, but >>> otherwise the assembler files are the same. Are you using the >>> cm3cg and cm3.cfg from my ftp site? >> >> Yes, downloaded a few hours ago. >> >> >>> Here is the version of the assembler that I have on my Intel Mac: >>> >>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>> assembler version 1.38 >> >> The mine is: >> >> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >> version 1.38 >> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >> > > Hmmm -- different versions of the assembler. Probably some flag > needs adjusting. > >> >> >>> >>> With respect to zeus, can you build with -verbose in the zeus >>> directory and send me the output? Sounds like one of the tools >>> is not getting built properly. >>> >> >> Ok, I am sending it as attachment >> >> >> > > Sorry, wrong option. Please run with -commands. I suspect stubgen > is failing for you. > > >> Renzo >> >> >>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>> >>>> You can find in the attachment the program and the compilation. >>>> >>>> Renzo >>>> >>>> >>>> >>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>> >>>>> >>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>> >>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>> >>>>>>> >>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>> >>>>>>>> Hello and lot of thanks! >>>>>>>> >>>>>>>> Everything is working really fine! >>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>> anonymous CVS), >>>>>>>> The process: >>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>> 3. running as su: >>>>>>>> do-cm3-core.sh buildship >>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>> do-cm3-std.sh buildship >>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>> and failing at the end the compilation of package zeus >>>>>>>> (which I suppose is an old issue) >>>>>>> >>>>>>> That's odd since my builds go through cleanly. >>>>>> >>>>>> This is an example of the strange "indirect jump" message: >>>>>> >>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>> --- building in I386_DARWIN --- >>>>>> new source -> compiling Main.m3 >>>>>> Main.ms:101:indirect jmp without `*' >>>>>> -> linking prog >>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>> Hello World >>>>>> >>>>>> maybe there is some problem with the linker parameters, >>>>>> however the progrum runs ok. >>>>> >>>>> Can you compile with "-keep" and send me the relevant .ms >>>>> file? I can compare with mine. >>>>> >>>>>> >>>>>> For what concerns zeus: the following is the output of cm3: >>>>>> >>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>> cvsstuff/cm3' +++ >>>>>> --- building in I386_DARWIN --- >>>>>> >>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> compilation failed => not building library "libm3zeus.a" >>>>>> Fatal Error: package build failed >>>>>> *** execution of failed *** >>>>>> >>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>> >>>>>> Several other packages cannot be compiled, like many obliq >>>>>> ones, webvt, June and mentor. >>>>>> >>>>>> However the compiler seems to work for compiling my stuff, >>>>>> altough I have to make a few corrections and debugging. >>>>> >>>>> This sounds problematic since it implies that the generators >>>>> for those files are not working correctly. (If you run with "- >>>>> commands" you'll see which generator programs are being used. >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Renzo >>>>>> >>>>>>> >>>>>>>> >>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>> century! >>>>>>>> >>>>>>>> Cordially, >>>>>>>> >>>>>>>> Renzo Orsini >>>>>>>> >>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>> >>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>> about 1G, by the way. >>>>>>>>> >>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>> >>>>>>>>>> What version of OSX are you using? >>>>>>>>>> >>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>> >>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>> >>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>> >>>>>>>>>>> - Darko >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>> >>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>> >>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/local/ >>>>>>>>>>>> cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>> this is the result: >>>>>>>>>>>> >>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>> CM3C = >>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh -c >>>>>>>>>>>> "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector >>>>>>>>>>>> digraph parseparams realgeometry set slisp >>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: pointer >>>>>>>>>>>> targets in passing argument 3 of 'accept' differ in >>>>>>>>>>>> signedness >>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>> errno=2 >>>>>>>>>>>> >>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>> install_file -- >>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>> >>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>> >>>>>>>>>>>> ====== >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I do >>>>>>>>>>>> not know if this is a normal behaviour. >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>> >>>>>>>>>>>> Cordially >>>>>>>>>>>> >>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>> >>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>> >>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>> >>>>>>>>>>>>> Darko. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo Orsini >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on an >>>>>>>>>>>>>>> intel mac, then >>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Since I would like to port the Fibonacci language for >>>>>>>>>>>>>>> databases, >>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>> documentation to start with. Of course I will be very >>>>>>>>>>>>>>> glad to give >>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>> >>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>> to install >>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>> be too >>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>> on it, >>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>> mail to >>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>> such an >>>>>>>>>>>>>> effort. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>> m3devel >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> M3devel mailing list >>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>> >>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>> ~hosking >>>>>>>>>> _--_|\ >>>>>>>>>> / \ >>>>>>>>>> \_.--._/ ) >>>>>>>>>> v / >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Antony Hosking | Associate Professor >>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>> ~hosking >>>>>>> _--_|\ >>>>>>> / \ >>>>>>> \_.--._/ ) >>>>>>> v / >>>>>>> >>>>>> >>>>> >>>>> Antony Hosking | Associate Professor >>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>> Purdue University | Mobile: (765) 427-5484 >>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>> _--_|\ >>>>> / \ >>>>> \_.--._/ ) >>>>> v / >>>>> >>>> >>> >>> Antony Hosking | Associate Professor >>> Dept of Computer Sciences | Office: (765) 494-6001 >>> Purdue University | Mobile: (765) 427-5484 >>> 250 N. University Street | hosking at cs.purdue.edu >>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>> _--_|\ >>> / \ >>> \_.--._/ ) >>> v / >>> >> > > Antony Hosking | Associate Professor > Dept of Computer Sciences | Office: (765) 494-6001 > Purdue University | Mobile: (765) 427-5484 > 250 N. University Street | hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Tue Apr 3 16:31:13 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 3 Apr 2007 10:31:13 -0400 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> Message-ID: <9753494C-96FE-4BDE-81AE-8CDDE9BC398B@cs.purdue.edu> -q also works. On Apr 3, 2007, at 9:33 AM, Darko wrote: > I'm not sure if this has been resolved in some more proper way, but > for the record, the warnings appearing after installation of the > XCode 2.4.1 tools can be gotten rid of by changing the following in > the config file: > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-o", object) > end > > so it reads > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-W -o", object) > end > > It's pretty obvious, but just in case anyone goes searching these > archives for an answer... > > > On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > >> On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: >> >>> >>> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >>> >>>> For some reason your .stabs entries are different than mine, but >>>> otherwise the assembler files are the same. Are you using the >>>> cm3cg and cm3.cfg from my ftp site? >>> >>> Yes, downloaded a few hours ago. >>> >>> >>>> Here is the version of the assembler that I have on my Intel Mac: >>>> >>>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>>> assembler version 1.38 >>> >>> The mine is: >>> >>> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >>> version 1.38 >>> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >>> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >>> >> >> Hmmm -- different versions of the assembler. Probably some flag >> needs adjusting. >> >>> >>> >>>> >>>> With respect to zeus, can you build with -verbose in the zeus >>>> directory and send me the output? Sounds like one of the tools >>>> is not getting built properly. >>>> >>> >>> Ok, I am sending it as attachment >>> >>> >>> >> >> Sorry, wrong option. Please run with -commands. I suspect >> stubgen is failing for you. >> >> >>> Renzo >>> >>> >>>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>>> >>>>> You can find in the attachment the program and the compilation. >>>>> >>>>> Renzo >>>>> >>>>> >>>>> >>>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>>> >>>>>> >>>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>>> >>>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>>> >>>>>>>> >>>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>>> >>>>>>>>> Hello and lot of thanks! >>>>>>>>> >>>>>>>>> Everything is working really fine! >>>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>>> anonymous CVS), >>>>>>>>> The process: >>>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>>> 3. running as su: >>>>>>>>> do-cm3-core.sh buildship >>>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>>> do-cm3-std.sh buildship >>>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>>> and failing at the end the compilation of package zeus >>>>>>>>> (which I suppose is an old issue) >>>>>>>> >>>>>>>> That's odd since my builds go through cleanly. >>>>>>> >>>>>>> This is an example of the strange "indirect jump" message: >>>>>>> >>>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>> --- building in I386_DARWIN --- >>>>>>> new source -> compiling Main.m3 >>>>>>> Main.ms:101:indirect jmp without `*' >>>>>>> -> linking prog >>>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>>> Hello World >>>>>>> >>>>>>> maybe there is some problem with the linker parameters, >>>>>>> however the progrum runs ok. >>>>>> >>>>>> Can you compile with "-keep" and send me the relevant .ms >>>>>> file? I can compare with mine. >>>>>> >>>>>>> >>>>>>> For what concerns zeus: the following is the output of cm3: >>>>>>> >>>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>>> cvsstuff/cm3' +++ >>>>>>> --- building in I386_DARWIN --- >>>>>>> >>>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> compilation failed => not building library "libm3zeus.a" >>>>>>> Fatal Error: package build failed >>>>>>> *** execution of failed *** >>>>>>> >>>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>>> >>>>>>> Several other packages cannot be compiled, like many obliq >>>>>>> ones, webvt, June and mentor. >>>>>>> >>>>>>> However the compiler seems to work for compiling my stuff, >>>>>>> altough I have to make a few corrections and debugging. >>>>>> >>>>>> This sounds problematic since it implies that the generators >>>>>> for those files are not working correctly. (If you run with "- >>>>>> commands" you'll see which generator programs are being used. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Renzo >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>>> century! >>>>>>>>> >>>>>>>>> Cordially, >>>>>>>>> >>>>>>>>> Renzo Orsini >>>>>>>>> >>>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>>> >>>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>>> about 1G, by the way. >>>>>>>>>> >>>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>>> >>>>>>>>>>> What version of OSX are you using? >>>>>>>>>>> >>>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>>> >>>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>>> >>>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>>> >>>>>>>>>>>> - Darko >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>>> >>>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>>> >>>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/ >>>>>>>>>>>>> local/cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>>> this is the result: >>>>>>>>>>>>> >>>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>>> CM3C = >>>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh - >>>>>>>>>>>>> c "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle >>>>>>>>>>>>> bitvector digraph parseparams realgeometry set slisp >>>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'accept' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>>> errno=2 >>>>>>>>>>>>> >>>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>>> install_file -- >>>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>>> >>>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>>> >>>>>>>>>>>>> ====== >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I >>>>>>>>>>>>> do not know if this is a normal behaviour. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>>> >>>>>>>>>>>>> Cordially >>>>>>>>>>>>> >>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Darko. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo >>>>>>>>>>>>>>> Orsini wrote: >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on >>>>>>>>>>>>>>>> an intel mac, then >>>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Since I would like to port the Fibonacci language >>>>>>>>>>>>>>>> for databases, >>>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>>> documentation to start with. Of course I will be >>>>>>>>>>>>>>>> very glad to give >>>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>>> to install >>>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>>> be too >>>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>>> on it, >>>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>>> mail to >>>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>>> such an >>>>>>>>>>>>>>> effort. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>>> m3devel >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>>> >>>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>>> ~hosking >>>>>>>>>>> _--_|\ >>>>>>>>>>> / \ >>>>>>>>>>> \_.--._/ ) >>>>>>>>>>> v / >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Antony Hosking | Associate Professor >>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>> ~hosking >>>>>>>> _--_|\ >>>>>>>> / \ >>>>>>>> \_.--._/ ) >>>>>>>> v / >>>>>>>> >>>>>>> >>>>>> >>>>>> Antony Hosking | Associate Professor >>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>>> _--_|\ >>>>>> / \ >>>>>> \_.--._/ ) >>>>>> v / >>>>>> >>>>> >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>> Purdue University | Mobile: (765) 427-5484 >>>> 250 N. University Street | hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>> >> >> Antony Hosking | Associate Professor >> Dept of Computer Sciences | Office: (765) 494-6001 >> Purdue University | Mobile: (765) 427-5484 >> 250 N. University Street | hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From stsp at elego.de Mon Apr 9 14:21:31 2007 From: stsp at elego.de (Stefan Sperling) Date: Mon, 9 Apr 2007 14:21:31 +0200 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461A2D05.6040208@elego.de> References: <46192AF7.6060406@elego.de> <20070408194650.GB32688@jack.stsp.lan> <4619836C.6060707@elego.de> <461A2D05.6040208@elego.de> Message-ID: <20070409122131.GB1350@ted.stsp.lan> On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels Janosch Hofmeyr wrote: > Just to complete the information in this thread: > ubuntu 6.10 does not have an asm/ipc.h. However, asm/ipc.h is not > necessary. It is sufficient to delete or comment the lines 19-21 in file > m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > #include > #endif > > This change has been committed to the repository, but was accidentally > not merged into the 5.4.0 release, apparently. Shall I create new source archives for the 5.4 branch that include this fix? If so, what version number should I put on them? 5.4.1? It should not be necessary to update the binary packages as well. Should I just copy them to a new name with the updated version number? -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From dabenavidesd at yahoo.es Mon Apr 9 21:47:35 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 9 Apr 2007 21:47:35 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409122131.GB1350@ted.stsp.lan> Message-ID: <636229.78700.qm@web27107.mail.ukl.yahoo.com> --- Stefan Sperling wrote: >On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels >Janosch Hofmeyr wrote: > Just to complete the information in this thread: > >It is sufficient to delete or comment > >the lines 19-21 in file > > > >m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > > #include > > #endif > > > > This change has been committed to the repository, > but was accidentally > > not merged into the 5.4.0 release, apparently. The problem was reported on the m3devel list on february 6, so It couldnt be merged on cm3-src-all-5.4.0.tgz file if that is the proeblem, that you refer. > Shall I create new source archives for the 5.4 > branch that include this > fix? If so, what version number should I put on > them? 5.4.1? I think this is the most appropiate to put a new file to download with the sources updated (not just that sysdeps.c update). > It should not be necessary to update the binary > packages as well. > Should I just copy them to a new name with the > updated version number? Antony, the bootstrap for the Pthread version is different from the bootstrap of cm3-5.4.0 release? I have exectuted do-cm3-std.sh buildship with the current cvs sources with cm3-5.4.0 bootstrap and in just one package I get a problem; Juno-2 with ubuntu 6.10, when tries to do a redirection of the of the PklFonts executable. ./PklFonts > FontData.pkl It got segmentation fault The thing is, after the rest of the system is compiled the Juno-2 compiles with no problems, including that redirection. ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From dabenavidesd at yahoo.es Tue Apr 10 01:49:25 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 01:49:25 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: Message-ID: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Hello: I have updated the libfreetype6 (to libfreetype6 2.2.1-5) library in ubuntu 6.10, and its now corrected the problem on juno with PklFonts, because, I have executed do-cm3-std.sh buildship with all def-std-pkg.sh packages and there was no problem when doing ./PklFonts > FontData.pkl However the Juno executable does not run, but this error shows: admin07 at sl07:~$ Juno *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "../src/runtime/common/RTCollector.m3", line 361 *** Cancelado admin07 at sl07:~$ Thanks --- "Daniel Alejandro Benavides D." wrote: > > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From hosking at cs.purdue.edu Tue Apr 10 02:15:59 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 9 Apr 2007 20:15:59 -0400 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> References: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Message-ID: <9ED98160-9379-4A83-951B-F07959ECDBEB@cs.purdue.edu> This looks like something wrong with your build. Please run "do-cm3- std.sh realclean" before "buildship". On Apr 9, 2007, at 7:49 PM, Daniel Alejandro Benavides D. wrote: > Hello: > I have updated the libfreetype6 (to libfreetype6 > 2.2.1-5) library in ubuntu 6.10, and its now corrected > the problem on juno with PklFonts, because, I have > executed do-cm3-std.sh buildship with all > def-std-pkg.sh packages and there was no problem when > doing ./PklFonts > FontData.pkl > > However the Juno executable does not run, but this > error shows: > > admin07 at sl07:~$ Juno > > > *** > *** runtime error: > *** An enumeration or subrange value was out of > range. > *** file "../src/runtime/common/RTCollector.m3", > line 361 > *** > > Cancelado > admin07 at sl07:~$ > > > Thanks > > --- "Daniel Alejandro Benavides D." > wrote: > >> >> >> I have exectuted do-cm3-std.sh buildship with the >> current cvs sources with cm3-5.4.0 bootstrap and in >> just one package I get a problem; Juno-2 with >> ubuntu >> 6.10, when tries to do a redirection of the of the >> PklFonts executable. >> ./PklFonts > FontData.pkl >> >> It got segmentation fault >> >> The thing is, after the rest of the system is >> compiled >> the Juno-2 compiles with no problems, including that >> redirection. >> >> > > > > ______________________________________________ > LLama Gratis a cualquier PC del Mundo. > Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. > http://es.voice.yahoo.com > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Tue Apr 10 04:30:46 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 04:30:46 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461AED47.8090906@elego.de> Message-ID: <20070410023046.3362.qmail@web27103.mail.ukl.yahoo.com> Hi: Can you check the version of libfreetype6? Mus be 2.2.1-5. The other you can do to check Juno-2 is cd to the path of juno and build it manually with cm3 -ship, in the right order of the directories: pkl-fonts juno-machine juno-compiler juno-app > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. Mentor does not depend on juno-2. What is the problem compiling mentor? Thanks --- Neels Janosch Hofmeyr wrote: > having gotten this useful information: > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > > I pondered a bit to find the right way of un-listing > juno-2. In the end > it turned out that I can comment lines 149 to 152 > from > scripts/def-std-pkgs.sh: > > # The Juno-2 graphical constraint based editor > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} pkl-fonts" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-machine" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-compiler" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-app" > > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. > > However, undoing the comments and compiling again > did not succeed. I > guess I'll have to live without the Juno-2 graphical > constraint based > editor... unless you have another suggestion, that > is. > > thanks, > neels > > Neels Janosch Hofmeyr wrote: > > Sorry, no success here... Do I really need this > juno thing? > > > > [...] > > === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > > +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > > -DROOT='/home/neels/elego/tmp/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > > Segmentation fault (core dumped) > > > "/home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile", > line > > 24: quake runtime error: exit 139: cd > ../pkl-fonts/LINUXLIBC6 && > > ./PklFonts > FontData.pkl > > > > --procedure-- -line- -file--- > > exec -- > > include_dir 24 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile > > 5 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/LINUXLIBC6/m3make.args > > > > Fatal Error: package build failed > > *** execution of failed *** > > > > $ env | grep LD > > LD_POINTER_GUARD=0 > > > > > > Stefan Sperling wrote: > > > >> On Mon, Apr 09, 2007 at 02:38:34PM +0200, Neels > Janosch Hofmeyr wrote: > >> > >> > >>> More problems have shown up with cm3 5.4.0 on my > ubuntu 6.10: > >>> > >>> " > >>> === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > >>> +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > >>> -DROOT='/home/neels/elego/tmp/cm3' +++ > >>> --- building in LINUXLIBC6 --- > >>> > >>> ignoring ../src/m3overrides > >>> > >>> cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > >>> Segmentation fault (core dumped) > >>> > >>> > >> Try this (from known problems page): > >> > >> Platforms: Fedora Core Linux, maybe other > distributions > >> > >> Description: All Modula3 programs (including cm3 > itself) exit with a > >> segmentation violation. > >> > >> Solution: Set the LD_POINTER_GUARD environment > variable to 0 before > >> running Modula3 programs, using a command like > "export > >> LD_POINTER_GUARD=0" or equivalent. > >> > >> > >> > > > > > > -- > Neels Janosch Hofmeyr > Software Developer > > neels at elego.de > Public Key: > http://binarchy.net/neels/neels.hofmeyr.public.key.asc > > elego Software Solutions GmbH > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > > ____________________________________________________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From darko at darko.org Tue Apr 10 21:01:54 2007 From: darko at darko.org (Darko) Date: Tue, 10 Apr 2007 21:01:54 +0200 Subject: [M3devel] Native M3 Implementation on Symbian Phones Message-ID: <07E52AF0-4E02-4AB5-88AE-41E4C1BC7F75@darko.org> It's interesting to note that Nokia have released 'OpenC' libraries < http://forum.nokia.com/openc > which seem to provide Posix like runtime and library support. There are limitations which may cause problems with the standard M3 runtime, but its certainly a step closer to being able to target such devices. Darko. From darko at darko.org Wed Apr 18 03:51:27 2007 From: darko at darko.org (Darko) Date: Wed, 18 Apr 2007 03:51:27 +0200 Subject: [M3devel] RTHooks CheckStoreTraced and CheckLoadTraced Message-ID: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Hi, Wondering if you can explain the use of these calls a little more. I'm currently using type maps to read and write fields from traced objects. Reading a traced reference from inside a traced object into a local variable is not working as it should. Should I use CheckLoadTraced and if so when and how? Looking at your changes to RTTypeMap, writing references into objects means you need to call CheckStoreTraced on the object written inside of, before it is written? Cheers, Darko. From wagner at plane.elego.de Wed Apr 18 08:04:18 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Wed, 18 Apr 2007 08:04:18 +0200 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] Message-ID: <20070418060417.GB14308@elegosoft.com> Is this the stack encryption problem again? Has anybody tried on Suse 10.1? Olaf ----- Forwarded message from Pino Zollo ----- DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= From: Pino Zollo To: m3-support at elego.de Subject: Installation Problem Date: Tue, 17 Apr 2007 18:00:50 -0400 X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no version=3.0.3 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Dear Sirs, I have installed the last version of cm3 but I have the following problem: cm3 compiles all programs, but are correctly executed only the ones compiled with the option build_standalone(). The programs which use shared libraries end in segmentation fault. Also the script do-cm3-std.sh fails to compile everything for the same problem: the program PklFonts in the directory m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault because needs shared libraries. Any suggestion ? I am using the distribution SuSE 10.1 with kernel 2.4.27 In the installation I have found that the command "export" was not doing properly its job, so I had to put the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the path in /etc/ld.so.conf I noticed that executables contain the string "/lib/ld-linux.so.2" so I had to make a symbolic link in /lib as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. I have also tried export "LD_POINTER_GUARD=0" also without any change. I have done quite a lot on Modula-3 programming years ago when kernel was 1.2.13. I would like to make public my old sources under the GNU licence. For this reason I am willing to check them on the new version on m3. I am available for doing development in Modula-3 in the limits of my capabilities. My name is Pino Zollo and my e-mail is pinozollo at gmail.com Some of my works are on http://www.qsl.net/zp4kfx and a description of my profesional experience is on http://www.qsl.net/zp4kfx/CV Thanks for the attencion. Best regards Pino ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 18 15:25:04 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 09:25:04 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > It seems I need a check on load of a traced reference in RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once I've checked it in (the CVS mailer is broken again I think). > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:34:58 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:34:58 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: I assume you have an apply method for your RTTypeMap.Visitor that takes "field: ADDRESS" and treats it as "REF REFANY". This is wrong. When reading a REF field you should use the following idiom: WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO ... access field via ref^ ... END; This will automatically insert a call to the appropriate runtime routines on accessing the reference field. There should be no need for you to call the runtime routines directly. On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:40:07 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:40:07 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: PS You should not use "REF REFANY", since a "REF" is assumed by the runtime system to be a *tidy* pointer to an object in the heap. Clearly, the address of a field inside an object is by definition *untidy*. On Apr 18, 2007, at 10:34 AM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:43:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:43:32 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <5E2DE156-C048-4E0B-A471-2A096BABA306@cs.purdue.edu> PS There is no fix to RTTypeMap needed here so long as you use the correct idiom as described earlier. This is unsafe code so you are expected to behave properly by the runtime system. On Apr 18, 2007, at 9:25 AM, Tony Hosking wrote: > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> > > It seems I need a check on load of a traced reference in > RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once > I've checked it in (the CVS mailer is broken again I think). > >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:50:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:50:08 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <5AB076A1-4E10-4BD0-8450-12E7B0F9B7B8@cs.purdue.edu> Does kernel 2.4 support NPTL? On Apr 18, 2007, at 2:04 AM, Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message- > id:from:to:subject:date:mime-version:content-type:content-transfer- > encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/ > WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQM > JJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ > U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 > tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on > plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on > plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following > problem: > > cm3 compiles all programs, but are correctly executed only the ones > compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not > doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did > put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" > so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any > change. > > I have done quite a lot on Modula-3 programming years ago when > kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to > check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a > description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney.bates at wichita.edu Wed Apr 18 18:15:58 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Wed, 18 Apr 2007 11:15:58 -0500 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <4626443E.8040704@wichita.edu> This symptom (works when statically linked, segfaults during startup when using dynamic libraries) is, as I remember, the symptom of the problem with changes in newer setjmp/longjmp that I posted about on January 11. The newer setjmp/longjmp mangle and demangle a couple of the registers (apparently to prevent some security exploit). When used as a pair, the change is harmless. But some uses of setjmp in the M3 libraries actually use the contents of the setjmp buffer to get register contents for thread switching, etc. There is an assembly-coded special version of setjmp in the M3 code, dating from many years back, that works with older longjmp, but not with the newest. Tony committed a change that removes this version of setjmp. but I don't think that fixes all the problems in all cases. I think existing uses of setjmp have to be split into two categories: those that just use the result to pass back to longjmp, and those that use the result for fetching registers. Then different variants of setjmp have to be used in the two cases. Antony suggested getcontext/setcontext, but I guess there are portability problems here? I looked at this briefly, and there are a very large number of uses of setjmp to check out. I have thought about working on it, but have been busy with other things for a while. I never got -DPTHREAD working and ran out of time on that approach. As I recall, -DPTHREAD is now the default, so I would guess Pino's installation is using it. Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following problem: > > cm3 compiles all programs, but are correctly executed only the ones compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any change. > > I have done quite a lot on Modula-3 programming years ago when kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Wed Apr 18 18:33:06 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 12:33:06 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <4626443E.8040704@wichita.edu> References: <20070418060417.GB14308@elegosoft.com> <4626443E.8040704@wichita.edu> Message-ID: Yes, use of setjmp/longjmp for *user*-level threads is bound to fail in the presence of the security mangling they now do, since new thread contexts are obtained by cloning a setjmp buffer from the root thread, and then hacking its stack pointer appropriately to match the new thread. This doesn't work with the secure setjmp/longjmp implementations. On platforms like Linux where setjmp/longjmp mangling prevent their use for user-level threading the only alternative is to use the explicit makecontext with getcontext/ setcontext. This is how *user*-level threading is done on Solaris. With the newer PTHREAD threading (for SOLgnu, SOLsun, LINUXLIBC6, I386_DARWIN, PPC_DARWIN) this should not be an issue, but it does require NPTL on Linux platforms (I don't think 2.4 kernels support NPTL). I do know that I can successfully build from the current CVS tree on recent releases of Fedora Core, without problems. On Apr 18, 2007, at 12:15 PM, Rodney M. Bates wrote: > This symptom (works when statically linked, segfaults during > startup when > using dynamic libraries) is, as I remember, the symptom of the > problem with > changes in newer setjmp/longjmp that I posted about on January 11. > > The newer setjmp/longjmp mangle and demangle a couple of the registers > (apparently to prevent some security exploit). When used as a > pair, the > change is harmless. But some uses of setjmp in the M3 libraries > actually > use the contents of the setjmp buffer to get register contents for > thread > switching, etc. > > There is an assembly-coded special version of setjmp in the M3 > code, dating > from many years back, that works with older longjmp, but not with > the newest. > Tony committed a change that removes this version of setjmp. but I > don't think > that fixes all the problems in all cases. I don't see problems with this change. What problems are you seeing? In this case, setjmp/longjmp is only being used for exception handling, assuming threads are using PTHREAD (i.e., Linux NPTL). > > I think existing uses of setjmp have to be split into two > categories: those > that just use the result to pass back to longjmp, and those that > use the result > for fetching registers. Then different variants of setjmp have to > be used in > the two cases. Antony suggested getcontext/setcontext, but I guess > there are > portability problems here? > > I looked at this briefly, and there are a very large number of uses > of setjmp > to check out. I have thought about working on it, but have been > busy with > other things for a while. > > I never got -DPTHREAD working and ran out of time on that approach. > As I recall, > -DPTHREAD is now the default, so I would guess Pino's installation > is using it. > > Olaf Wagner wrote: >> Is this the stack encryption problem again? Has anybody tried on >> Suse 10.1? >> Olaf >> ----- Forwarded message from Pino Zollo ----- >> DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; >> d=gmail.com; s=beta; >> h=domainkey-signature:received:received:message- >> id:from:to:subject:date:mime-version:content-type:content-transfer- >> encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; >> b=U78B+w8R6V2/ >> WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQ >> MJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ >> U4PrfCQw+NQLqPP2k0Vwncv7Ss= >> From: Pino Zollo >> To: m3-support at elego.de >> Subject: Installation Problem >> Date: Tue, 17 Apr 2007 18:00:50 -0400 >> X-Spam-Status: No, score=2.7 required=5.0 >> tests=DEAR_SOMETHING,RCVD_BY_IP, >> RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no >> version=3.0.3 >> X-Spam-Level: ** >> X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on >> plane.elego.de >> X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on >> plane.elego.de >> X-Virus-Status: Clean >> Dear Sirs, >> I have installed the last version of cm3 but I have the following >> problem: >> cm3 compiles all programs, but are correctly executed only the >> ones compiled >> with the option build_standalone(). >> The programs which use shared libraries end in segmentation fault. >> Also the script do-cm3-std.sh fails to compile everything for the >> same >> problem: the program PklFonts in the directory >> m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault >> because needs shared libraries. >> Any suggestion ? >> I am using the distribution SuSE 10.1 with kernel 2.4.27 >> In the installation I have found that the command "export" was not >> doing >> properly its job, so I had to put >> the path /usr/local/cm3/bin in /etc/profile. For libraries I did >> put the >> path in /etc/ld.so.conf >> I noticed that executables contain the string "/lib/ld-linux.so.2" >> so I had >> to make a symbolic link in /lib >> as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. >> I have also tried export "LD_POINTER_GUARD=0" also without any >> change. >> I have done quite a lot on Modula-3 programming years ago when >> kernel was >> 1.2.13. I would like to make public my >> old sources under the GNU licence. For this reason I am willing to >> check >> them on the new version on m3. >> I am available for doing development in Modula-3 in the limits of my >> capabilities. >> My name is Pino Zollo and my e-mail is pinozollo at gmail.com >> Some of my works are on http://www.qsl.net/zp4kfx and a >> description of my >> profesional experience is on >> http://www.qsl.net/zp4kfx/CV >> Thanks for the attencion. >> Best regards >> Pino >> ----- End forwarded message ----- > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From darko at darko.org Thu Apr 19 03:37:10 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 03:37:10 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Not actually using that code, but I did see your helpful commit on it some time ago. It seems that the problem I described earlier is due to another bug. The code I'm working on does a lot of reading and writing of traced references and I was hoping to get a better understanding of situations I need to be aware of and how to use those RTHooks calls more efficiently. So say I have two traced objects and I am copying a traced reference from a field in one to the other. What set of calls would be normally required there? I'm sure you haven't got time to go into the minutiae of the runtime system, but any hints would be appreciated. Cheers, Darko. On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > From hosking at cs.purdue.edu Thu Apr 19 04:05:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:05:42 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. You SHOULD NOT need to use these calls so long as you are accessing using properly typed references (as in my example earlier). This is not an issue for SAFE code, only in UNSAFE modules must you be careful how you use LOOPHOLE to cast references. > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? How are you copying the references? I would like to see your code for this. So long as you properly type things then the calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be generated by the compiler. Anyway... RTHooks.CheckLoadTracedRef is generated by the compiler whenever you access a traced reference in memory (e.g., via dereference or from a shared variable). Its job is to prevent mutators from acquiring references to "gray" objects (that are reachable but still to be scanned for the traced references that they contain). The check turns the object from gray to black by scanning its references and making sure none of them refer to white objects (that are not known to be reachable by the collector). RTHooks.CheckStoreTraced is generated by the compiler whenever a traced reference is stored (or might be stored, e.g., for VAR) into a traced object. The check makes sure that the generational GC knows that the object has been modified. It needs to know this for objects in the older genration. Again, I reiterate that it is not my intention that programmers actually call these runtime hooks explicitly! I am certain I can rewrite any code you might have so that you do not need to call them explicitly. Instead, properly typed references will result in the checks being generated for you by the compiler. Please feel free to send me code snippets for review. > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From hosking at cs.purdue.edu Thu Apr 19 04:09:35 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:09:35 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <55BF1964-E58E-4E20-93F7-9DDA1A7B1CF4@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. > > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? If you have: VAR source, target: OBJECT field: REFANY END; then copying field from source to target is written as normal: target.field := source.field; The compiler will emit the appropriate calls: CheckLoadTracedRef(source.field) CheckStoreTraced(target) Why do you need to call these runtime routines explicitly if the compiler will generate them for you? > > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From darko at darko.org Thu Apr 19 04:49:29 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 04:49:29 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> Message-ID: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Thanks, that's very helpful. I agree I shouldn't need it but it's exactly because I'm not using the compiler is why I need to know about it. The code is attached, it's still in development and unfinished but you'll get the idea. Basically the module allows you to programatically access structure fields. It allows for applications like being able to create indexes (using an adaptation of the Table code) of objects using arbitrary fields as keys. It makes dynamic programming a bit more dynamic. The code is intended to be completely safe, but there's more checks I need to do. -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.m3 Type: application/octet-stream Size: 41052 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.i3 Type: application/octet-stream Size: 6117 bytes Desc: not available URL: -------------- next part -------------- On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > > On Apr 18, 2007, at 9:37 PM, Darko wrote: > >> Not actually using that code, but I did see your helpful commit on >> it some time ago. It seems that the problem I described earlier is >> due to another bug. The code I'm working on does a lot of reading >> and writing of traced references and I was hoping to get a better >> understanding of situations I need to be aware of and how to use >> those RTHooks calls more efficiently. > > You SHOULD NOT need to use these calls so long as you are accessing > using properly typed references (as in my example earlier). This > is not an issue for SAFE code, only in UNSAFE modules must you be > careful how you use LOOPHOLE to cast references. > >> So say I have two traced objects and I am copying a traced >> reference from a field in one to the other. What set of calls >> would be normally required there? > > How are you copying the references? I would like to see your code > for this. So long as you properly type things then the calls to > RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be > generated by the compiler. > > Anyway... > > RTHooks.CheckLoadTracedRef is generated by the compiler whenever > you access a traced reference in memory (e.g., via dereference or > from a shared variable). Its job is to prevent mutators from > acquiring references to "gray" objects (that are reachable but > still to be scanned for the traced references that they contain). > The check turns the object from gray to black by scanning its > references and making sure none of them refer to white objects > (that are not known to be reachable by the collector). > > RTHooks.CheckStoreTraced is generated by the compiler whenever a > traced reference is stored (or might be stored, e.g., for VAR) into > a traced object. The check makes sure that the generational GC > knows that the object has been modified. It needs to know this for > objects in the older genration. > > Again, I reiterate that it is not my intention that programmers > actually call these runtime hooks explicitly! I am certain I can > rewrite any code you might have so that you do not need to call > them explicitly. Instead, properly typed references will result in > the checks being generated for you by the compiler. Please feel > free to send me code snippets for review. > >> I'm sure you haven't got time to go into the minutiae of the >> runtime system, but any hints would be appreciated. >> >> Cheers, >> Darko. >> >> >> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >> >>> I assume you have an apply method for your RTTypeMap.Visitor that >>> takes "field: ADDRESS" and treats it as "REF REFANY". This is >>> wrong. When reading a REF field you should use the following idiom: >>> >>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>> ... access field via ref^ ... >>> END; >>> >>> This will automatically insert a call to the appropriate runtime >>> routines on accessing the reference field. >>> >>> There should be no need for you to call the runtime routines >>> directly. >>> >>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>> >>>> Hi, >>>> >>>> Wondering if you can explain the use of these calls a little >>>> more. I'm currently using type maps to read and write fields >>>> from traced objects. Reading a traced reference from inside a >>>> traced object into a local variable is not working as it should. >>>> Should I use CheckLoadTraced and if so when and how? Looking at >>>> your changes to RTTypeMap, writing references into objects means >>>> you need to call CheckStoreTraced on the object written inside >>>> of, before it is written? >>>> >>>> Cheers, >>>> Darko. >>> > From hosking at cs.purdue.edu Thu Apr 19 15:34:40 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 09:34:40 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Message-ID: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> My immediate reaction is that you probably want to approach reads slightly differently. Instead of using your generic InTypeRead routine to access the appropriate bits/bytes of the value, you probably want to replace it with some sort of simple addressing mechanism that computes the raw address of the field. Traced reference fields must always be aligned within the heap object they are stored in so your addressing for these will not need bit offsets. Once you have an ADDRESS "addr" for the traced reference field, so long as you use the idiom I gave earlier, you will get the correct behavior for reads. That is, WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO .. use field^ to access the traced reference stored at address field .. END; The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the traced reference "r" once you have loopholed it through the raw field access into a REFANY, before you return it from InTypeReadRef: PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): REFANY = VAR v: REFANY; BEGIN this.check(field, Kind.Ref, obj); this.read(obj, field, ADR(v), BYTESIZE(v)); RTHooks.CheckLoadTracedRef(v); RETURN v; END InTypeReadRef; This is safe for the current CM3 ambiguous roots collector, but this code would probably break with a fully copying/relocating collector (if one was ever implemented for Modula-3), since the raw bytes of the reference might become stale between the point you load them from memory and the point they are placed in the typed REFANY variable "v" (effectively, you'd be hiding a reference from the GC for a brief period, and it would not be able to update that hidden reference in the face of its target being moved). I note, generally, that your field addressing mechanisms only work anyway because we are using an ambiguous roots collector that pins all objects referenced from the thread stacks (your "obj" parameters in the accessor methods), which allows the use of RTHeap.GetDataAdr to provide an address for the object that is valid while the object reference is held on the stack. When writing references to object fields you will need to invoke "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is being stored. Thus, for InTypeWriteRef: PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: CARDINAL) = BEGIN this.check(field, Kind.Ref, obj); WITH f = this.offs.get(field) DO IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; END; this.write(obj, field, ADR(val), BYTESIZE(val)); RTHooks.CheckStoreTraced(obj); END InTypeWriteRef; If you are performing multiple writes of reference fields on the same object (e.g., to an array) then you can invoke CheckStoreTraced once for the whole object. Hope this helps. On Apr 18, 2007, at 10:49 PM, Darko wrote: > Thanks, that's very helpful. I agree I shouldn't need it but it's > exactly because I'm not using the compiler is why I need to know > about it. The code is attached, it's still in development and > unfinished but you'll get the idea. > > Basically the module allows you to programatically access structure > fields. It allows for applications like being able to create > indexes (using an adaptation of the Table code) of objects using > arbitrary fields as keys. It makes dynamic programming a bit more > dynamic. The code is intended to be completely safe, but there's > more checks I need to do. > > > > > > > > > > On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > >> >> On Apr 18, 2007, at 9:37 PM, Darko wrote: >> >>> Not actually using that code, but I did see your helpful commit >>> on it some time ago. It seems that the problem I described >>> earlier is due to another bug. The code I'm working on does a lot >>> of reading and writing of traced references and I was hoping to >>> get a better understanding of situations I need to be aware of >>> and how to use those RTHooks calls more efficiently. >> >> You SHOULD NOT need to use these calls so long as you are >> accessing using properly typed references (as in my example >> earlier). This is not an issue for SAFE code, only in UNSAFE >> modules must you be careful how you use LOOPHOLE to cast references. >> >>> So say I have two traced objects and I am copying a traced >>> reference from a field in one to the other. What set of calls >>> would be normally required there? >> >> How are you copying the references? I would like to see your code >> for this. So long as you properly type things then the calls to >> RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be >> generated by the compiler. >> >> Anyway... >> >> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >> you access a traced reference in memory (e.g., via dereference or >> from a shared variable). Its job is to prevent mutators from >> acquiring references to "gray" objects (that are reachable but >> still to be scanned for the traced references that they contain). >> The check turns the object from gray to black by scanning its >> references and making sure none of them refer to white objects >> (that are not known to be reachable by the collector). >> >> RTHooks.CheckStoreTraced is generated by the compiler whenever a >> traced reference is stored (or might be stored, e.g., for VAR) >> into a traced object. The check makes sure that the generational >> GC knows that the object has been modified. It needs to know this >> for objects in the older genration. >> >> Again, I reiterate that it is not my intention that programmers >> actually call these runtime hooks explicitly! I am certain I can >> rewrite any code you might have so that you do not need to call >> them explicitly. Instead, properly typed references will result >> in the checks being generated for you by the compiler. Please >> feel free to send me code snippets for review. >> >>> I'm sure you haven't got time to go into the minutiae of the >>> runtime system, but any hints would be appreciated. >>> >>> Cheers, >>> Darko. >>> >>> >>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>> >>>> I assume you have an apply method for your RTTypeMap.Visitor >>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>> This is wrong. When reading a REF field you should use the >>>> following idiom: >>>> >>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>> ... access field via ref^ ... >>>> END; >>>> >>>> This will automatically insert a call to the appropriate runtime >>>> routines on accessing the reference field. >>>> >>>> There should be no need for you to call the runtime routines >>>> directly. >>>> >>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>> >>>>> Hi, >>>>> >>>>> Wondering if you can explain the use of these calls a little >>>>> more. I'm currently using type maps to read and write fields >>>>> from traced objects. Reading a traced reference from inside a >>>>> traced object into a local variable is not working as it >>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>> Looking at your changes to RTTypeMap, writing references into >>>>> objects means you need to call CheckStoreTraced on the object >>>>> written inside of, before it is written? >>>>> >>>>> Cheers, >>>>> Darko. >>>> >> > From darko at darko.org Thu Apr 19 19:09:45 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 19:09:45 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: Thanks, thats very useful. Garbage collection is a bit of a mystery, since I can't imagine myself changing it I've never studied it, but it's on my mind constantly. M3 is so safe whenever something weird happens with my code I know it's because I've trodden on a traced reference, or copied it into a untraced space and back or something. I've relied on the pinning of references that appear in the stack extensively, and I think a lot of other code does too. It a bit of a worry to think that it's possible it might disappear. If we don't have this, short of stopping the collector, what do you do if you want your object to stay still? On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > My immediate reaction is that you probably want to approach reads > slightly differently. Instead of using your generic InTypeRead > routine to access the appropriate bits/bytes of the value, you > probably want to replace it with some sort of simple addressing > mechanism that computes the raw address of the field. Traced > reference fields must always be aligned within the heap object they > are stored in so your addressing for these will not need bit > offsets. Once you have an ADDRESS "addr" for the traced reference > field, so long as you use the idiom I gave earlier, you will get > the correct behavior for reads. That is, > > WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO > .. use field^ to access the traced reference stored at address > field .. > END; > > The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the > traced reference "r" once you have loopholed it through the raw > field access into a REFANY, before you return it from InTypeReadRef: > > PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): > REFANY = > VAR v: REFANY; > BEGIN > this.check(field, Kind.Ref, obj); > this.read(obj, field, ADR(v), BYTESIZE(v)); > RTHooks.CheckLoadTracedRef(v); > RETURN v; > END InTypeReadRef; > > This is safe for the current CM3 ambiguous roots collector, but > this code would probably break with a fully copying/relocating > collector (if one was ever implemented for Modula-3), since the raw > bytes of the reference might become stale between the point you > load them from memory and the point they are placed in the typed > REFANY variable "v" (effectively, you'd be hiding a reference from > the GC for a brief period, and it would not be able to update that > hidden reference in the face of its target being moved). > > I note, generally, that your field addressing mechanisms only work > anyway because we are using an ambiguous roots collector that pins > all objects referenced from the thread stacks (your "obj" > parameters in the accessor methods), which allows the use of > RTHeap.GetDataAdr to provide an address for the object that is > valid while the object reference is held on the stack. > > When writing references to object fields you will need to invoke > "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is > being stored. Thus, for InTypeWriteRef: > > PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: > CARDINAL) = > BEGIN > this.check(field, Kind.Ref, obj); > WITH f = this.offs.get(field) DO > IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE > (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; > END; > this.write(obj, field, ADR(val), BYTESIZE(val)); > RTHooks.CheckStoreTraced(obj); > END InTypeWriteRef; > > If you are performing multiple writes of reference fields on the > same object (e.g., to an array) then you can invoke > CheckStoreTraced once for the whole object. > > Hope this helps. > > On Apr 18, 2007, at 10:49 PM, Darko wrote: > >> Thanks, that's very helpful. I agree I shouldn't need it but it's >> exactly because I'm not using the compiler is why I need to know >> about it. The code is attached, it's still in development and >> unfinished but you'll get the idea. >> >> Basically the module allows you to programatically access >> structure fields. It allows for applications like being able to >> create indexes (using an adaptation of the Table code) of objects >> using arbitrary fields as keys. It makes dynamic programming a bit >> more dynamic. The code is intended to be completely safe, but >> there's more checks I need to do. >> >> >> >> >> >> >> >> >> >> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >> >>> >>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>> >>>> Not actually using that code, but I did see your helpful commit >>>> on it some time ago. It seems that the problem I described >>>> earlier is due to another bug. The code I'm working on does a >>>> lot of reading and writing of traced references and I was hoping >>>> to get a better understanding of situations I need to be aware >>>> of and how to use those RTHooks calls more efficiently. >>> >>> You SHOULD NOT need to use these calls so long as you are >>> accessing using properly typed references (as in my example >>> earlier). This is not an issue for SAFE code, only in UNSAFE >>> modules must you be careful how you use LOOPHOLE to cast references. >>> >>>> So say I have two traced objects and I am copying a traced >>>> reference from a field in one to the other. What set of calls >>>> would be normally required there? >>> >>> How are you copying the references? I would like to see your >>> code for this. So long as you properly type things then the >>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>> will be generated by the compiler. >>> >>> Anyway... >>> >>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>> you access a traced reference in memory (e.g., via dereference or >>> from a shared variable). Its job is to prevent mutators from >>> acquiring references to "gray" objects (that are reachable but >>> still to be scanned for the traced references that they >>> contain). The check turns the object from gray to black by >>> scanning its references and making sure none of them refer to >>> white objects (that are not known to be reachable by the collector). >>> >>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>> traced reference is stored (or might be stored, e.g., for VAR) >>> into a traced object. The check makes sure that the generational >>> GC knows that the object has been modified. It needs to know >>> this for objects in the older genration. >>> >>> Again, I reiterate that it is not my intention that programmers >>> actually call these runtime hooks explicitly! I am certain I can >>> rewrite any code you might have so that you do not need to call >>> them explicitly. Instead, properly typed references will result >>> in the checks being generated for you by the compiler. Please >>> feel free to send me code snippets for review. >>> >>>> I'm sure you haven't got time to go into the minutiae of the >>>> runtime system, but any hints would be appreciated. >>>> >>>> Cheers, >>>> Darko. >>>> >>>> >>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>> >>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>> This is wrong. When reading a REF field you should use the >>>>> following idiom: >>>>> >>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>> ... access field via ref^ ... >>>>> END; >>>>> >>>>> This will automatically insert a call to the appropriate >>>>> runtime routines on accessing the reference field. >>>>> >>>>> There should be no need for you to call the runtime routines >>>>> directly. >>>>> >>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Wondering if you can explain the use of these calls a little >>>>>> more. I'm currently using type maps to read and write fields >>>>>> from traced objects. Reading a traced reference from inside a >>>>>> traced object into a local variable is not working as it >>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>> objects means you need to call CheckStoreTraced on the object >>>>>> written inside of, before it is written? >>>>>> >>>>>> Cheers, >>>>>> Darko. >>>>> >>> >> > From dragisha at m3w.org Thu Apr 19 19:24:57 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 19:24:57 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's Message-ID: <1177003497.11459.17.camel@faramir.m3w.org> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box (Pentium M). It breaks somewhere where PklFonts is to be called. I am not sure if it's because of built binary being started w/ my glibc, or some other issue. Is cm3-5.4.0 possible to build on box like this? Thanks, dd -- Dragi?a Duri? From hosking at cs.purdue.edu Thu Apr 19 19:39:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:39:55 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: On Apr 19, 2007, at 1:09 PM, Darko wrote: > Thanks, thats very useful. Garbage collection is a bit of a > mystery, since I can't imagine myself changing it I've never > studied it, but it's on my mind constantly. M3 is so safe whenever > something weird happens with my code I know it's because I've > trodden on a traced reference, or copied it into a untraced space > and back or something. Yeah, it can be a problem, but if you are careful it should not be too hard to avoid. The idea is not to "hide" a reference where the GC cannot find it. > > I've relied on the pinning of references that appear in the stack > extensively, and I think a lot of other code does too. It a bit of > a worry to think that it's possible it might disappear. If we don't > have this, short of stopping the collector, what do you do if you > want your object to stay still? In the absence of implicit pinning of references from the stack we would need to introduce an explicit pin/unpin API for programmers to use. It is unlikely that CM3 will ever move away from implicit pinning unless we were to build a "smart" backend compiler (ie, not based on gcc). So, short answer is that you are probably OK for a long time to come. > > > On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > >> My immediate reaction is that you probably want to approach reads >> slightly differently. Instead of using your generic InTypeRead >> routine to access the appropriate bits/bytes of the value, you >> probably want to replace it with some sort of simple addressing >> mechanism that computes the raw address of the field. Traced >> reference fields must always be aligned within the heap object >> they are stored in so your addressing for these will not need bit >> offsets. Once you have an ADDRESS "addr" for the traced reference >> field, so long as you use the idiom I gave earlier, you will get >> the correct behavior for reads. That is, >> >> WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO >> .. use field^ to access the traced reference stored at address >> field .. >> END; >> >> The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on >> the traced reference "r" once you have loopholed it through the >> raw field access into a REFANY, before you return it from >> InTypeReadRef: >> >> PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): >> REFANY = >> VAR v: REFANY; >> BEGIN >> this.check(field, Kind.Ref, obj); >> this.read(obj, field, ADR(v), BYTESIZE(v)); >> RTHooks.CheckLoadTracedRef(v); >> RETURN v; >> END InTypeReadRef; >> >> This is safe for the current CM3 ambiguous roots collector, but >> this code would probably break with a fully copying/relocating >> collector (if one was ever implemented for Modula-3), since the >> raw bytes of the reference might become stale between the point >> you load them from memory and the point they are placed in the >> typed REFANY variable "v" (effectively, you'd be hiding a >> reference from the GC for a brief period, and it would not be able >> to update that hidden reference in the face of its target being >> moved). >> >> I note, generally, that your field addressing mechanisms only work >> anyway because we are using an ambiguous roots collector that pins >> all objects referenced from the thread stacks (your "obj" >> parameters in the accessor methods), which allows the use of >> RTHeap.GetDataAdr to provide an address for the object that is >> valid while the object reference is held on the stack. >> >> When writing references to object fields you will need to invoke >> "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field >> is being stored. Thus, for InTypeWriteRef: >> >> PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: >> CARDINAL) = >> BEGIN >> this.check(field, Kind.Ref, obj); >> WITH f = this.offs.get(field) DO >> IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE >> (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; >> END; >> this.write(obj, field, ADR(val), BYTESIZE(val)); >> RTHooks.CheckStoreTraced(obj); >> END InTypeWriteRef; >> >> If you are performing multiple writes of reference fields on the >> same object (e.g., to an array) then you can invoke >> CheckStoreTraced once for the whole object. >> >> Hope this helps. >> >> On Apr 18, 2007, at 10:49 PM, Darko wrote: >> >>> Thanks, that's very helpful. I agree I shouldn't need it but it's >>> exactly because I'm not using the compiler is why I need to know >>> about it. The code is attached, it's still in development and >>> unfinished but you'll get the idea. >>> >>> Basically the module allows you to programatically access >>> structure fields. It allows for applications like being able to >>> create indexes (using an adaptation of the Table code) of objects >>> using arbitrary fields as keys. It makes dynamic programming a >>> bit more dynamic. The code is intended to be completely safe, but >>> there's more checks I need to do. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >>> >>>> >>>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>>> >>>>> Not actually using that code, but I did see your helpful commit >>>>> on it some time ago. It seems that the problem I described >>>>> earlier is due to another bug. The code I'm working on does a >>>>> lot of reading and writing of traced references and I was >>>>> hoping to get a better understanding of situations I need to be >>>>> aware of and how to use those RTHooks calls more efficiently. >>>> >>>> You SHOULD NOT need to use these calls so long as you are >>>> accessing using properly typed references (as in my example >>>> earlier). This is not an issue for SAFE code, only in UNSAFE >>>> modules must you be careful how you use LOOPHOLE to cast >>>> references. >>>> >>>>> So say I have two traced objects and I am copying a traced >>>>> reference from a field in one to the other. What set of calls >>>>> would be normally required there? >>>> >>>> How are you copying the references? I would like to see your >>>> code for this. So long as you properly type things then the >>>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>>> will be generated by the compiler. >>>> >>>> Anyway... >>>> >>>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>>> you access a traced reference in memory (e.g., via dereference >>>> or from a shared variable). Its job is to prevent mutators from >>>> acquiring references to "gray" objects (that are reachable but >>>> still to be scanned for the traced references that they >>>> contain). The check turns the object from gray to black by >>>> scanning its references and making sure none of them refer to >>>> white objects (that are not known to be reachable by the >>>> collector). >>>> >>>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>>> traced reference is stored (or might be stored, e.g., for VAR) >>>> into a traced object. The check makes sure that the >>>> generational GC knows that the object has been modified. It >>>> needs to know this for objects in the older genration. >>>> >>>> Again, I reiterate that it is not my intention that programmers >>>> actually call these runtime hooks explicitly! I am certain I >>>> can rewrite any code you might have so that you do not need to >>>> call them explicitly. Instead, properly typed references will >>>> result in the checks being generated for you by the compiler. >>>> Please feel free to send me code snippets for review. >>>> >>>>> I'm sure you haven't got time to go into the minutiae of the >>>>> runtime system, but any hints would be appreciated. >>>>> >>>>> Cheers, >>>>> Darko. >>>>> >>>>> >>>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>>> >>>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>>> This is wrong. When reading a REF field you should use the >>>>>> following idiom: >>>>>> >>>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>>> ... access field via ref^ ... >>>>>> END; >>>>>> >>>>>> This will automatically insert a call to the appropriate >>>>>> runtime routines on accessing the reference field. >>>>>> >>>>>> There should be no need for you to call the runtime routines >>>>>> directly. >>>>>> >>>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Wondering if you can explain the use of these calls a little >>>>>>> more. I'm currently using type maps to read and write fields >>>>>>> from traced objects. Reading a traced reference from inside a >>>>>>> traced object into a local variable is not working as it >>>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>>> objects means you need to call CheckStoreTraced on the object >>>>>>> written inside of, before it is written? >>>>>>> >>>>>>> Cheers, >>>>>>> Darko. >>>>>> >>>> >>> >> From hosking at cs.purdue.edu Thu Apr 19 19:41:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:41:08 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177003497.11459.17.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: I just bootstrapped cm3 from the CVS head last night, on the most recent up-to-date Fedora core, so I don't know what your problems could be. I wonder if you have a broken bootstrap compiler? On Apr 19, 2007, at 1:24 PM, Dragi?a Duri? wrote: > I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > (Pentium > M). It breaks somewhere where PklFonts is to be called. I am not > sure if > it's because of built binary being started w/ my glibc, or some other > issue. > > Is cm3-5.4.0 possible to build on box like this? > > Thanks, > dd > > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Thu Apr 19 19:55:10 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Thu, 19 Apr 2007 19:55:10 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: <20070419175510.GA16758@elegosoft.com> On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > I just bootstrapped cm3 from the CVS head last night, on the most > recent up-to-date Fedora core, so I don't know what your problems > could be. I wonder if you have a broken bootstrap compiler? > > On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > > >I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >(Pentium > >M). It breaks somewhere where PklFonts is to be called. I am not > >sure if > >it's because of built binary being started w/ my glibc, or some other > >issue. > > > >Is cm3-5.4.0 possible to build on box like this? Hallo Antony, could you provide a minimal binary installation package for LINUXLIBC6 with PTHREADs that we could put on our server for download? I assume many people will run into problems on Linux with the current release. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Thu Apr 19 22:27:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 16:27:42 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419175510.GA16758@elegosoft.com> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: >> I just bootstrapped cm3 from the CVS head last night, on the most >> recent up-to-date Fedora core, so I don't know what your problems >> could be. I wonder if you have a broken bootstrap compiler? >> >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: >> >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box >>> (Pentium >>> M). It breaks somewhere where PklFonts is to be called. I am not >>> sure if >>> it's because of built binary being started w/ my glibc, or some >>> other >>> issue. >>> >>> Is cm3-5.4.0 possible to build on box like this? > > Hallo Antony, > > could you provide a minimal binary installation package for LINUXLIBC6 > with PTHREADs that we could put on our server for download? I assume > many people will run into problems on Linux with the current release. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Thu Apr 19 23:01:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 23:01:20 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: <1177016480.4282.13.camel@faramir.m3w.org> I will try this one... But I am not sure we understand each other here... Bootstrap compiler from download site works for me, except for problems with PklFonts and mentor, also met by other ppl... I will look in that sometimes, as Trestle apps are not of primary interest for me anyway... Problem happens when trying to run some executable, I'll paste fisheye example... faramir:dragisha/pts/7: ~/cm3-inst% fisheye zsh: segmentation fault fisheye It is what happens. I must fix it with: faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye Is this only way to run cm3 built executables under modern Linuces? dd On Thu, 2007-04-19 at 16:27 -0400, Tony Hosking wrote: > There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > > > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > >> I just bootstrapped cm3 from the CVS head last night, on the most > >> recent up-to-date Fedora core, so I don't know what your problems > >> could be. I wonder if you have a broken bootstrap compiler? > >> > >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > >> > >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >>> (Pentium > >>> M). It breaks somewhere where PklFonts is to be called. I am not > >>> sure if > >>> it's because of built binary being started w/ my glibc, or some > >>> other > >>> issue. > >>> > >>> Is cm3-5.4.0 possible to build on box like this? > > > > Hallo Antony, > > > > could you provide a minimal binary installation package for LINUXLIBC6 > > with PTHREADs that we could put on our server for download? I assume > > many people will run into problems on Linux with the current release. > > > > Olaf > > -- > > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > > Berlin, Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > > 23 45 86 95 > > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > > Sitz: Berlin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > > DE163214194 > -- Dragi?a Duri? From stsp at elego.de Thu Apr 19 23:05:09 2007 From: stsp at elego.de (Stefan Sperling) Date: Thu, 19 Apr 2007 23:05:09 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177016480.4282.13.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> Message-ID: <20070419210509.GB11507@jack.stsp.lan> On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: > Problem happens when trying to run some executable, I'll paste fisheye > example... > > faramir:dragisha/pts/7: ~/cm3-inst% fisheye > zsh: segmentation fault fisheye > > It is what happens. I must fix it with: > > faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye > > Is this only way to run cm3 built executables under modern Linuces? Yes. It's because of a change in recent glibc versions. I forgot the specifics, I guess someone else will have to elaborate :-P -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From hosking at cs.purdue.edu Thu Apr 19 23:44:20 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 17:44:20 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419210509.GB11507@jack.stsp.lan> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: Download and unzip the bootstrap compiler executable from ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. Rebuild using this bootstrap compiler from the most recent CVS sources checked out from elegosoft. This will build a pthreads-enabled run-time system that avoids the pointer guard problems with user-threading based on setjmp/longjmp. I verified this build for me just last night on the most recent release of Fedora Core. On Apr 19, 2007, at 5:05 PM, Stefan Sperling wrote: > On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: >> Problem happens when trying to run some executable, I'll paste >> fisheye >> example... >> >> faramir:dragisha/pts/7: ~/cm3-inst% fisheye >> zsh: segmentation fault fisheye >> >> It is what happens. I must fix it with: >> >> faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye >> >> Is this only way to run cm3 built executables under modern Linuces? > > Yes. It's because of a change in recent glibc versions. > I forgot the specifics, I guess someone else will have to > elaborate :-P > -- > stefan > http://stsp.name PGP Key: > 0xF59D25F0 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Fri Apr 20 07:59:02 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Fri, 20 Apr 2007 07:59:02 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: <20070420055902.GA1418@elegosoft.com> On Thu, Apr 19, 2007 at 05:44:20PM -0400, Tony Hosking wrote: > Download and unzip the bootstrap compiler executable from ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > Rebuild using this bootstrap compiler from the most recent CVS > sources checked out from elegosoft. > > This will build a pthreads-enabled run-time system that avoids the > pointer guard problems with user-threading based on setjmp/longjmp. > > I verified this build for me just last night on the most recent > release of Fedora Core. Hi Stefan, I just checked our known problem pages, and the problem with the segment violation in indeed already documented. Could you additionally put up Antony's short description together with his link for a current CM3 bootstrap? Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 09:25:38 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 09:25:38 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. Message-ID: <1177053938.4282.29.camel@faramir.m3w.org> While maintaining our spinoff pm3 version, I've cleaned most of syscall wrapper needs from library source. Problem during early m3core development was, most probably, unclean situation about where and how C library will be invoked and someone decided to play safe, and to wrap all syscalls implied by used library calls to be fully safe. Across m3core and libm3, number of these low level, C library calls is really small. File I/O, socket I/O, filesystem operations... make a big majority of them (more than 98%). As all of these low level C lib calls these were easily identifiable (few grep's), I've cleaned them without much problems and after that, I've eliminated wrapper by wrapper. At this moment, I think none is present in our in-house m3 env. As time permits, and unless none else makes these steps, I will repeat them for CM3. I know answer to this following question is hidden somewhere in archives, but as my previous questions has shown - it can be hidden very deep :). So, can someone highlight current m3gc and syscall wrappers situation for me? How do I build CM3 WITH most extensive GC available at the moment? dd P.S. Big thanks to Tony Hosking for LD_POINTER_GUARD! As I have many Modula-3 apps in production use, this was very big issue for me. -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 10:07:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 10:07:20 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles Message-ID: <1177056440.4282.34.camel@faramir.m3w.org> while trying to make my programs with newsly installed cm3, I am constantly running into same error. Example: import("libm3") include_dir("../../../../m3lib0/src") include_dir("../../../../pdf/src") include_dir("../../../../zip/src") implementation("Test") program("test") Is my m3makefile, and when "cm3" is run in package's src dir, I have only: faramir:dragisha/pts/9: test/1/src% cm3 --- building in ../LINUXLIBC6 --- *** *** runtime error: *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES list *** file "../src/os/POSIX/PathnamePosix.m3", line 98 *** Mentioned line raises this FATAL exception when Absolute(path) is TRUE... What is absolute in my paths? All m3makefiles in question were ok with pm3, of course.. dd -- Dragi?a Duri? From wagner at elegosoft.com Fri Apr 20 11:57:34 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 11:57:34 +0200 (CEST) Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <1177053938.4282.29.camel@faramir.m3w.org> References: <1177053938.4282.29.camel@faramir.m3w.org> Message-ID: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: > While maintaining our spinoff pm3 version, I've cleaned most of syscall > wrapper needs from library source. Problem during early m3core > development was, most probably, unclean situation about where and how C > library will be invoked and someone decided to play safe, and to wrap > all syscalls implied by used library calls to be fully safe. > > Across m3core and libm3, number of these low level, C library calls is > really small. File I/O, socket I/O, filesystem operations... make a big > majority of them (more than 98%). As all of these low level C lib calls > these were easily identifiable (few grep's), I've cleaned them without > much problems and after that, I've eliminated wrapper by wrapper. At > this moment, I think none is present in our in-house m3 env. You would also need to consider all indirect system calls via libraries and embedded C code. I think the M3 developers tried to offer a more or less complete set of safe system calls, regardless how they are called from the M3 program. > As time > permits, and unless none else makes these steps, I will repeat them for > CM3. I don't think this is a good idea right now. > I know answer to this following question is hidden somewhere in > archives, but as my previous questions has shown - it can be hidden very > deep :). So, can someone highlight current m3gc and syscall wrappers > situation for me? How do I build CM3 WITH most extensive GC available at > the moment? The current CM3 compiler/runtime does not need virtual memory protection and thus does not need the system call wrappers any more, as Antony has modified the compiler to generate hints for the garbage collector. So you can more or less ignore this issue (unless you really want memory protection) and simply use m3gc-simple for all platforms. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 14:21:19 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:19 +0200 Subject: [M3devel] cm3 pthread problem? Message-ID: <1177071680.4282.47.camel@faramir.m3w.org> Attached program won't run under freshly built cm3 @ Fedora Core 6. Compiles, runs... and stucks... It works flawlessly with user-level threads and also with mine implementation of NPTL for pm3. I've done it for 4000 threads per 10000 passess, and this version I am attaching, and trying without success on CM3 is 40 threads... Of course, before I run it, I am ulimiting stack size to 64kB or similar.. It fits. dd -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 14:21:56 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:56 +0200 Subject: [M3devel] missing attachment, sorry Message-ID: <1177071717.4282.49.camel@faramir.m3w.org> -- Dragi?a Duri? -------------- next part -------------- MODULE TestThreads EXPORTS Main; IMPORT Fmt, IO, Thread; TYPE MyClosure = Thread.Closure OBJECT my, next: Thread.Condition; OVERRIDES apply := JustDoIt; END; VAR glob := NEW(MUTEX); started: CARDINAL; PROCEDURE JustDoIt(c: MyClosure): REFANY = BEGIN LOCK glob DO INC(started); Thread.Wait(glob, c.my); Thread.Signal(c.next); END; RETURN NIL; END JustDoIt; CONST nThreads = 40; VAR my, first, a, b: Thread.Condition; BEGIN FOR j:=1 TO 10 DO IO.Put("Forking " & Fmt.Int(nThreads) & " threads... (" & Fmt.Int(j) & ")\n"); IF j MOD 10 = 0 THEN IO.Put("(" & Fmt.Int(j) & ")"); END; my := NEW(Thread.Condition); a:=NEW(Thread.Condition); started:=0; FOR i:=1 TO nThreads DO IF i=nThreads THEN b:=my; ELSE b:=NEW(Thread.Condition); END; EVAL Thread.Fork(NEW(MyClosure, my := a, next := b)); IF i=1 THEN first:=a; END; a:=b; END; IO.Put("Forked.\n"); IO.Put(","); WHILE started References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> Message-ID: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 1:10 pm, Dragi??a Duri?? wrote: > On Fri, 2007-04-20 at 11:57 +0200, Olaf Wagner wrote: >> On Fri, April 20, 2007 9:25 am, Dragi? ??a Duri????? wrote: >> > While maintaining our spinoff pm3 version, I've cleaned most of syscall >> > wrapper needs from library source. Problem during early m3core >> > development was, most probably, unclean situation about where and how C >> > library will be invoked and someone decided to play safe, and to wrap >> > all syscalls implied by used library calls to be fully safe. >> > >> > Across m3core and libm3, number of these low level, C library calls is >> > really small. File I/O, socket I/O, filesystem operations... make a big >> > majority of them (more than 98%). As all of these low level C lib calls >> > these were easily identifiable (few grep's), I've cleaned them without >> > much problems and after that, I've eliminated wrapper by wrapper. At >> > this moment, I think none is present in our in-house m3 env. >> >> You would also need to consider all indirect system calls via libraries >> and embedded C code. I think the M3 developers tried to offer a more or >> less complete set of safe system calls, regardless how they are called >> from the M3 program. > > It is, but C libraries are mostly in these few percents remaining... > Good wrapping techniques and auditing of existing libraries (hint: grep > EXTERNAL) would be small effort for seamless integration of full m3 > runtime with unperfect C world. I was trying to say that cover only the calls from libraries now used by the M3 runtime, but M3 is an open language and allows the integration of C and C++ (and other) code (and thus of arbitrary other libraries). >> > I know answer to this following question is hidden somewhere in >> > archives, but as my previous questions has shown - it can be hidden very >> > deep :). So, can someone highlight current m3gc and syscall wrappers >> > situation for me? How do I build CM3 WITH most extensive GC available at >> > the moment? >> >> The current CM3 compiler/runtime does not need virtual memory protection >> and thus does not need the system call wrappers any more, as Antony has >> modified the compiler to generate hints for the garbage collector. So >> you can more or less ignore this issue (unless you really want memory >> protection) and simply use m3gc-simple for all platforms. > > Does it mean garbage collection is not > incremental/generational/supercallifraglistic... anymore, or Anthony's > work has made it work without memory protection? The latter; we've got incremental and generational garbage collection without the need for memory protection and system call wrappers :) Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Fri Apr 20 14:59:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 08:59:32 -0400 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> Message-ID: <9B2640F3-10E7-4CE6-BB63-32090198FA1C@cs.purdue.edu> Yes, I concur that there is no need to expend any effort on syscall wrappers. I plan to eliminate the legacy syscall wrappers from all CM3 targets very soon, since the new GC no longer needs them. I would hate to see the library sources cluttered with unnecessary changes that relate only to the deprecated syscall wrappers. On Apr 20, 2007, at 5:57 AM, Olaf Wagner wrote: > > On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: >> While maintaining our spinoff pm3 version, I've cleaned most of >> syscall >> wrapper needs from library source. Problem during early m3core >> development was, most probably, unclean situation about where and >> how C >> library will be invoked and someone decided to play safe, and to wrap >> all syscalls implied by used library calls to be fully safe. >> >> Across m3core and libm3, number of these low level, C library >> calls is >> really small. File I/O, socket I/O, filesystem operations... make >> a big >> majority of them (more than 98%). As all of these low level C lib >> calls >> these were easily identifiable (few grep's), I've cleaned them >> without >> much problems and after that, I've eliminated wrapper by wrapper. At >> this moment, I think none is present in our in-house m3 env. > > You would also need to consider all indirect system calls via > libraries > and embedded C code. I think the M3 developers tried to offer a > more or > less complete set of safe system calls, regardless how they are called > from the M3 program. > >> As time >> permits, and unless none else makes these steps, I will repeat >> them for >> CM3. > > I don't think this is a good idea right now. > >> I know answer to this following question is hidden somewhere in >> archives, but as my previous questions has shown - it can be >> hidden very >> deep :). So, can someone highlight current m3gc and syscall wrappers >> situation for me? How do I build CM3 WITH most extensive GC >> available at >> the moment? > > The current CM3 compiler/runtime does not need virtual memory > protection > and thus does not need the system call wrappers any more, as Antony > has > modified the compiler to generate hints for the garbage collector. So > you can more or less ignore this issue (unless you really want memory > protection) and simply use m3gc-simple for all platforms. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Fri Apr 20 15:06:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 15:06:11 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> Message-ID: <1177074372.4282.54.camel@faramir.m3w.org> On Fri, 2007-04-20 at 14:56 +0200, Olaf Wagner wrote: ... > >> You would also need to consider all indirect system calls via libraries > >> and embedded C code. I think the M3 developers tried to offer a more or > >> less complete set of safe system calls, regardless how they are called > >> from the M3 program. > > > > It is, but C libraries are mostly in these few percents remaining... > > Good wrapping techniques and auditing of existing libraries (hint: grep > > EXTERNAL) would be small effort for seamless integration of full m3 > > runtime with unperfect C world. > > I was trying to say that cover only the calls from libraries now used > by the M3 runtime, but M3 is an open language and allows the integration > of C and C++ (and other) code (and thus of arbitrary other libraries). It is exactly what I was pointing to... Arbitrary other libraries will have defined entry points from Modula-3 code, and that is what matters... What and how C library code is doing with _its data_ is not of our concern. What is it doing with passed data concerns us, and it is a place where interfacing guidelines/policies kick in. ... > > Does it mean garbage collection is not > > incremental/generational/supercallifraglistic... anymore, or Anthony's > > work has made it work without memory protection? > > The latter; we've got incremental and generational garbage collection > without the need for memory protection and system call wrappers :) Oh. Oh. Ohhhhhhhhh. :) GREAT! > > Olaf -- Dragi?a Duri? From hosking at cs.purdue.edu Fri Apr 20 15:08:51 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:08:51 -0400 Subject: [M3devel] cm3 pthread problem? In-Reply-To: <1177071680.4282.47.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> Message-ID: I'm investigating this issue. Looks like some sort of race causing deadlock. On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: > Attached program won't run under freshly built cm3 @ Fedora Core 6. > Compiles, runs... and stucks... > > It works flawlessly with user-level threads and also with mine > implementation of NPTL for pm3. I've done it for 4000 threads per > 10000 > passess, and this version I am attaching, and trying without > success on > CM3 is 40 threads... > > Of course, before I run it, I am ulimiting stack size to 64kB or > similar.. It fits. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Fri Apr 20 15:16:33 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:16:33 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <1177056440.4282.34.camel@faramir.m3w.org> References: <1177056440.4282.34.camel@faramir.m3w.org> Message-ID: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> I have my hands full with GC evolution and thread issues at the moment. Anyone else care to look into this? On Apr 20, 2007, at 4:07 AM, Dragi?a Duri? wrote: > while trying to make my programs with newsly installed cm3, I am > constantly running into same error. Example: > > import("libm3") > > include_dir("../../../../m3lib0/src") > include_dir("../../../../pdf/src") > include_dir("../../../../zip/src") > > implementation("Test") > program("test") > > Is my m3makefile, and when "cm3" is run in package's src dir, I have > only: > > faramir:dragisha/pts/9: test/1/src% cm3 > --- building in ../LINUXLIBC6 --- > > > > *** > *** runtime error: > *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > list > *** file "../src/os/POSIX/PathnamePosix.m3", line 98 > *** > > Mentioned line raises this FATAL exception when Absolute(path) is > TRUE... What is absolute in my paths? > > All m3makefiles in question were ok with pm3, of course.. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at elegosoft.com Fri Apr 20 15:38:53 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 15:38:53 +0200 (CEST) Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <57472.194.138.127.36.1177076333.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 3:16 pm, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I can take this one, but won't be able to look into it until tonight (no M3 here at my current customer work), and if I don't find it fast, it will be delayed to Monday evening. If this is too late or anybody else can be faster, please let me know. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > >> while trying to make my programs with newsly installed cm3, I am >> constantly running into same error. Example: >> >> import("libm3") >> >> include_dir("../../../../m3lib0/src") >> include_dir("../../../../pdf/src") >> include_dir("../../../../zip/src") >> >> implementation("Test") >> program("test") >> >> Is my m3makefile, and when "cm3" is run in package's src dir, I have >> only: >> >> faramir:dragisha/pts/9: test/1/src% cm3 >> --- building in ../LINUXLIBC6 --- >> >> >> >> *** >> *** runtime error: >> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >> list >> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >> *** >> >> Mentioned line raises this FATAL exception when Absolute(path) is >> TRUE... What is absolute in my paths? >> >> All m3makefiles in question were ok with pm3, of course.. >> >> dd >> -- >> Dragi??a Duri?? >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 21:36:25 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 21:36:25 +0200 Subject: [M3devel] order of module initialization... Message-ID: <1177097786.4282.75.camel@faramir.m3w.org> I remember debugging this first time we tried to migrate to CM3... And then we solved it, for current CM3 and current version of our software, mentioning some imported module in EXPORTS clause... at the moment, this linked these modules in initialization chain and it worked ok... Right now, it does not. Module XLBuiltin is one imported regularly, and XLModuleUDP (for example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ wrong moment... What would be solution for this? dd -- Dragi?a Duri? From jayk123 at hotmail.com Fri Apr 20 23:58:55 2007 From: jayk123 at hotmail.com (j k) Date: Fri, 20 Apr 2007 21:58:55 +0000 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: Reminds me of C++ globals with constructors. The solution is to avoid initialization. Do everything "on demand" and hide all state behind functions. and then "on demand" needs to be thread safe, so there is a small bootstrapping problem -- basically the only initialization you should do, speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more accurately, nothing "in the same layer", if you have a dependency graph, only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are (unless you are working on ntdll.dll, but even then.. :) ) If you don't have a strict layered design, or can't capture what it is and analyze statically, again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an easy way to be correct. And even then, you can often avoid the critical sections via something like InterlockedCompareExchangePointer, even implement spin locks for this rare code. (Hand written spin locks are generally to be avoided since the kernel won't know about them and you won't schedule efficiently, whereas with EnterCriticalSection, if there is contention, the kernel has a hint what is going on and can schedule better). Don't have any "public global data", unless, and I don't know if this applies in Modula-3, unless it is "constant initialized", which I'm not sure is well defined in C++ or not. I do believe it is well defined in C..except dynamic linking I think screws that up to. (pardon the Win32-centricity; I'm sure the principles apply more broadly) Ok, in C++ and in Modula-3, there are maybe better answers. In C++, globals within a source file are guaranteed to be initialized in the order they occur in the file. But the order of files is not defined. There is a documented trick..potentially very inefficient..that I don't have the patience to describe here.. "the iostream counter trick"... this I don't think has an analog in Modula-3, but sure. Stroupstroup's Design and Evolution book documents it. - Jay >From: Dragi??a Duri?? >To: "M3 Developers' Mailing List" >Subject: [M3devel] order of module initialization... >Date: Fri, 20 Apr 2007 21:36:25 +0200 > >I remember debugging this first time we tried to migrate to CM3... And >then we solved it, for current CM3 and current version of our software, >mentioning some imported module in EXPORTS clause... at the moment, this >linked these modules in initialization chain and it worked ok... Right >now, it does not. > >Module XLBuiltin is one imported regularly, and XLModuleUDP (for >example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >wrong moment... > >What would be solution for this? > >dd >-- >Dragi??a Duri?? > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _________________________________________________________________ Download Messenger. Join the i?m Initiative. Help make a difference today. http://im.live.com/messenger/im/home/?source=TAGHM_APR07 From mika at async.caltech.edu Sat Apr 21 00:15:16 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 20 Apr 2007 15:15:16 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Fri, 20 Apr 2007 21:58:55 -0000." Message-ID: <200704202215.l3KMFGY4013501@camembert.async.caltech.edu> The order in Modula-3 is defined in section 2.5.6 of the language report (page 47 of Systems Programming with Modula-3). The problem described here appears to be that EXPORTS isn't treated the same as IMPORT (which it should be). There are some other problems I noticed when porting PM3 code to CM3, don't know if they have been fixed. The "solution" in that case was to IMPORT interfaces in places where they weren't really needed. Obfuscating one's code to avoid taking advantage of the convenience of module initialization seems like a rather severe suggestion... The rule, by the way, is: "If module M depends on module N and N does not depend on M, then N's body will be executed before M's body, where: * A module M 'depends on' a module N if M uses an interface that N exports or if M depends on a module that depends on N. * A module M 'uses' an interface I if M imports or exports I or if M uses an interface that imports I. Except for this constraint, the order of execution is implementation-dependent." Mika "j k" writes: >Reminds me of C++ globals with constructors. > >The solution is to avoid initialization. > >Do everything "on demand" and hide all state behind functions. > and then "on demand" needs to be thread safe, so there is a small >bootstrapping problem -- basically the only initialization you should do, >speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >accurately, nothing "in the same layer", if you have a dependency graph, >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are >(unless you are working on ntdll.dll, but even then.. :) ) If you don't have >a strict layered design, or can't capture what it is and analyze statically, >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an >easy way to be correct. > >And even then, you can often avoid the critical sections via something like >InterlockedCompareExchangePointer, even implement spin locks for this rare >code. (Hand written spin locks are generally to be avoided since the kernel >won't know about them and you won't schedule efficiently, whereas with >EnterCriticalSection, if there is contention, the kernel has a hint what is >going on and can schedule better). > >Don't have any "public global data", unless, and I don't know if this >applies in Modula-3, unless it is "constant initialized", which I'm not sure >is well defined in C++ or not. I do believe it is well defined in C..except >dynamic linking I think screws that up to. > >(pardon the Win32-centricity; I'm sure the principles apply more broadly) > >Ok, in C++ and in Modula-3, there are maybe better answers. >In C++, globals within a source file are guaranteed to be initialized in the >order they occur in the file. >But the order of files is not defined. >There is a documented trick..potentially very inefficient..that I don't have >the patience to describe here.. "the iostream counter trick"... this I don't >think has an analog in Modula-3, but sure. >Stroupstroup's Design and Evolution book documents it. > >- Jay > > >>From: Dragi??a Duri?? >>To: "M3 Developers' Mailing List" >>Subject: [M3devel] order of module initialization... >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >>I remember debugging this first time we tried to migrate to CM3... And >>then we solved it, for current CM3 and current version of our software, >>mentioning some imported module in EXPORTS clause... at the moment, this >>linked these modules in initialization chain and it worked ok... Right >>now, it does not. >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >>wrong moment... >> >>What would be solution for this? >> >>dd >>-- >>Dragi??a Duri?? >> >>_______________________________________________ >>M3devel mailing list >>M3devel at elegosoft.com >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Download Messenger. Join the i?m Initiative. Help make a difference today. >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Sun Apr 22 09:29:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 22 Apr 2007 09:29:11 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> Message-ID: <1177226951.737.1.camel@faramir.m3w.org> It looks like this fix never came through... With cvs-head CM3, unpickling of TextRefTbl.T instance still does not work. dd On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > Rodney, > > I've checked in fixes to RTTypeMap and clients that should restore > functional pickling support. For some reason my commit messages are > awaiting moderator approval (it seems my commit id is not the same as > my subscription to the m3commit list) before they can be sent out. > Please make sure that unpickling now works for you. > > Best regards, > > Tony > > On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > > > I tracked down a bug unpickling to the following. When executing > > in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > > > > WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > > ref^ := v.readRef(nelem.refType); > > END; > > > > has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > > CheckStoreTraced expects ref to be a normal reference to an object, > > computes the address of its header (4 bytes less), and sets the > > dirty bit. > > > > But ConvertPacking is reading in to a field of a recently created > > object that is not first in the object. It computes ref to point > > to the field, and CheckStoreTraced actually steps on the previous > > real field instead of the header. > > > > It looks like the actions of CheckStoreTraced are needed, but I > > don't right off hand see how to recode Convert to fix this. It's > > use of ref can't be moved. Presumably, there could be other unsafe > > code elsewhere using the same technique to store data. > > > > Does the compiler need a different criterion on when to generate > > the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > > still is pointing inside a traced object. Maybe people who modify > > objects in this way need to explicitly call CheckStoreTraced? > > > > -- > > ------------------------------------------------------------- > > Rodney M. Bates, retired assistant professor > > Dept. of Computer Science, Wichita State University > > Wichita, KS 67260-0083 > > 316-978-3922 > > rodney.bates at wichita.edu > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > Antony Hosking | Associate Professor > Dept of Computer Science | Office: +1 765 494-6001 > Purdue University | Mobile: +1 765 427-5484 > 250 N. University Street | Email: hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:22:11 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:22:11 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177226951.737.1.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: Yes, this fix is in CVS head. On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > It looks like this fix never came through... With cvs-head CM3, > unpickling of TextRefTbl.T instance still does not work. > > dd > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >> Rodney, >> >> I've checked in fixes to RTTypeMap and clients that should restore >> functional pickling support. For some reason my commit messages are >> awaiting moderator approval (it seems my commit id is not the same as >> my subscription to the m3commit list) before they can be sent out. >> Please make sure that unpickling now works for you. >> >> Best regards, >> >> Tony >> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >> >>> I tracked down a bug unpickling to the following. When executing >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>> >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>> ref^ := v.readRef(nelem.refType); >>> END; >>> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). >>> CheckStoreTraced expects ref to be a normal reference to an object, >>> computes the address of its header (4 bytes less), and sets the >>> dirty bit. >>> >>> But ConvertPacking is reading in to a field of a recently created >>> object that is not first in the object. It computes ref to point >>> to the field, and CheckStoreTraced actually steps on the previous >>> real field instead of the header. >>> >>> It looks like the actions of CheckStoreTraced are needed, but I >>> don't right off hand see how to recode Convert to fix this. It's >>> use of ref can't be moved. Presumably, there could be other unsafe >>> code elsewhere using the same technique to store data. >>> >>> Does the compiler need a different criterion on when to generate >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>> still is pointing inside a traced object. Maybe people who modify >>> objects in this way need to explicitly call CheckStoreTraced? >>> >>> -- >>> ------------------------------------------------------------- >>> Rodney M. Bates, retired assistant professor >>> Dept. of Computer Science, Wichita State University >>> Wichita, KS 67260-0083 >>> 316-978-3922 >>> rodney.bates at wichita.edu >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> Antony Hosking | Associate Professor >> Dept of Computer Science | Office: +1 765 494-6001 >> Purdue University | Mobile: +1 765 427-5484 >> 250 N. University Street | Email: hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:47:50 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:47:50 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177233124.737.4.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> Message-ID: <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> It is a pthreads call that is failing: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; I'm guessing we are in a race with a thread that is in the middle of dying (i.e., the error code, "r", is "ESRCH: thread is an invalid thread ID"). Can you confirm that is the error? As for why it is happening I don't know, since the thread handle is only ever accessed while activeMu is locked, which enforces atomicity of the thread's presence in the ring of live threads along with its handle being valid: WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; IF allThreads = me THEN allThreads := me.next; END; me.next.prev := me.prev; me.prev.next := me.next; me.next := NIL; me.prev := NIL; WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; I wonder if this is a system limit on the number of threads or some such (but then why would the pthread_create call not return an error?) Can you diagnose further? On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: > After upping it a bit, with stack limit set to 64k and 4000 > threads in > each loop... Is not happening at same place every run, but most of > runs > - it happens. > > Forking 4000 threads... (1) > Forked. > ,: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 > *** > > > On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >> I have fixed this bug and checked in the fix to CVS (why are we not >> seeing the commit messages again?). It did turn out to be a deadlock >> issue in the handshake between allocating threads and the GC. Your >> sample program now runs with no problems on my dual-core Intel Mac. >> Please let me know if you have any other threading difficulties. >> >> Regards, >> >> Tony >> >> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >> >>> Attached program won't run under freshly built cm3 @ Fedora Core 6. >>> Compiles, runs... and stucks... >>> >>> It works flawlessly with user-level threads and also with mine >>> implementation of NPTL for pm3. I've done it for 4000 threads per >>> 10000 >>> passess, and this version I am attaching, and trying without >>> success on >>> CM3 is 40 threads... >>> >>> Of course, before I run it, I am ulimiting stack size to 64kB or >>> similar.. It fits. >>> >>> dd >>> -- >>> Dragi?a Duri? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:59:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:59:46 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177253233.737.18.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> Message-ID: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > Hello, > > Have been skimming through source of PTHREAD code... And I think > job can > be done without so much relying on how-they-did-it-before, esp with > regard to list of waiters and similar internal and global structures. > Also, I see number of global locks and I am sure they are congestion > generators every now and while, esp in heavy threading situations. > > Of course, there is number of approaches to this multi-thread > situations. Mine being one of very nonconservative use of threads, I > think it is important to remain open to possibly very big number of > threads running in single process - meaning scalability is one of > primary objections... As global locks don't do well with > scalability, I > don't like "cm" and similar global congestion points. Yes, there are tensions between a thin/absent veneer between language threads and system threads. Most important are issues of preserving a reasonable memory model for programmers (see Hans Boehm's paper http://portal.acm.org/citation.cfm?doid=1065010.1065042). There are also questions of portability and debuggability. I agree that global locks are to be avoided where they cause known contention, but there are tradeoffs there too. For large numbers of threads (as you appear to need) I think we would need to adopt some other implementation approach, possibly by multiplexing multiple lightweight user-level threads on some smaller number of heavyweight system-level threads, but then you run into scheduling and load-balancing problems. > We talked about this at least once before, and I think I remember you > insisted on more compatibility than can be read from SPwM3. Do you > think > best idea would be to integrate mine NPTL code into CM3 for people to > trash and test, and let everyone select what is best for their > situation? What I wanted was a situation where programs would be able to run with the same tools (e.g., showheap, showthread) under both user- level and system threading. This goal has been achieved with the current pthread-based implementation. Moreover, I wanted something where variations in thread support from one system to another could be exploited most easily (such as for systems where thread suspend/ resume is provided as a primitive). Again, the current implementation achieves this, and runs with minimal target-specific code on Darwin, Solaris, and Linux. Ports to other targets should be relatively straightforward. > Problems I had with my pthread implementation were all related to VM > hell of earlier GC implementation... After you did that piece of art > with new approach to GC, I expect infinite uptimes from my servers and > bots :). Big thanks for that! Any native threading implementation is going to have problems with VM- based memory management. I'm surprised to were able to get anything going with the VM-based GC. > > dd > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 23:50:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 17:50:46 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177275494.737.37.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> <1177275494.737.37.camel@faramir.m3w.org> Message-ID: <8C54780A-5D27-4239-92BE-1D274340BE5A@cs.purdue.edu> Argh! That signal isn't even documented as being returned by pthread_kill on Linux (at least in my man pages). So the solution appears to be to replace: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; with: LOOP WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO IF r = 0 THEN EXIT; END; IF r # Uerror.EAGAIN THEN <*ASSERT FALSE*>; END; (* try it again *) END; END; What do you think? I wonder how many other places I should be checking for EAGAIN? On Apr 22, 2007, at 4:58 PM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:47 -0400, Tony Hosking wrote: >> It is a pthreads call that is failing: >> >> WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO >> <*ASSERT r=0*> >> END; >> >> I'm guessing we are in a race with a thread that is in the middle of >> dying (i.e., the error code, "r", is "ESRCH: thread is an invalid >> thread ID"). Can you confirm that is the error? > > No :). It's 11 - EAGAIN, IIRC. Lot's of signals are sent there, and it > kicked some signal dequeuing limit, or something like. > >> As for why it is happening I don't know, since the thread handle is >> only ever accessed while activeMu is locked, which enforces atomicity >> of the thread's presence in the ring of live threads along with its >> handle being valid: >> >> WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; >> IF allThreads = me THEN allThreads := me.next; END; >> me.next.prev := me.prev; >> me.prev.next := me.next; >> me.next := NIL; >> me.prev := NIL; >> WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; >> WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; >> >> I wonder if this is a system limit on the number of threads or some >> such (but then why would the pthread_create call not return an >> error?) Can you diagnose further? > > Limit is not, it's sure. I am running test in parallel with mine > implementation + pm3, and it works without a flaw, 10000 rounds, 4000 > threads each. I am using some realtime signal for suspend magic, > though... Don't remember right now, but I think there is something > special about this queuing of realtime and non-realtime signals... > >> >> On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: >> >>> After upping it a bit, with stack limit set to 64k and 4000 >>> threads in >>> each loop... Is not happening at same place every run, but most of >>> runs >>> - it happens. >>> >>> Forking 4000 threads... (1) >>> Forked. >>> ,: >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 >>> *** >>> >>> >>> On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >>>> I have fixed this bug and checked in the fix to CVS (why are we not >>>> seeing the commit messages again?). It did turn out to be a >>>> deadlock >>>> issue in the handshake between allocating threads and the GC. Your >>>> sample program now runs with no problems on my dual-core Intel Mac. >>>> Please let me know if you have any other threading difficulties. >>>> >>>> Regards, >>>> >>>> Tony >>>> >>>> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >>>> >>>>> Attached program won't run under freshly built cm3 @ Fedora >>>>> Core 6. >>>>> Compiles, runs... and stucks... >>>>> >>>>> It works flawlessly with user-level threads and also with mine >>>>> implementation of NPTL for pm3. I've done it for 4000 threads per >>>>> 10000 >>>>> passess, and this version I am attaching, and trying without >>>>> success on >>>>> CM3 is 40 threads... >>>>> >>>>> Of course, before I run it, I am ulimiting stack size to 64kB or >>>>> similar.. It fits. >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 00:00:03 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:00:03 -0400 Subject: [M3devel] Fwd: A GC/compiler/Pickle interaction bug References: <1177277134.737.47.camel@faramir.m3w.org> Message-ID: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> This works just fine for me on the most recent CVS head for I386_DARWIN. I wonder if you have not updated your compiler to match? There was a matching fix to the compiler. You should build and ship: m3gc-simple m3core libm3 m3middle m3linker m3front m3quake cm3 Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your cm3.cfg). This will get you a new compiler. Then, rebuild your CM3 installation from the CVS head. Begin forwarded message: > From: Dragi?a Duri? > Date: April 22, 2007 5:25:33 PM EDT > To: Tony Hosking > Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug > > On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >> Can you send me a stack dump from gdb at the point of the error? >> Even better, do you have a simple example program I can use to test >> with? > > Here is one. Quick and dirty, but it shows problem. > > -- > Dragi?a Duri? -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Pickles.m3 URL: -------------- next part -------------- From hosking at cs.purdue.edu Mon Apr 23 00:01:05 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:01:05 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> Message-ID: <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> PS The bootstrap compilers on elegosoft probably may to be rebuilt for this fix to take effect. On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: > This works just fine for me on the most recent CVS head for > I386_DARWIN. > > I wonder if you have not updated your compiler to match? There was > a matching fix to the compiler. You should build and ship: > > m3gc-simple > m3core > libm3 > m3middle > m3linker > m3front > m3quake > cm3 > > Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your > cm3.cfg). This will get you a new compiler. > > Then, rebuild your CM3 installation from the CVS head. > > Begin forwarded message: > >> From: Dragi?a Duri? >> Date: April 22, 2007 5:25:33 PM EDT >> To: Tony Hosking >> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >> >> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>> Can you send me a stack dump from gdb at the point of the error? >>> Even better, do you have a simple example program I can use to test >>> with? >> >> Here is one. Quick and dirty, but it shows problem. >> >> -- >> Dragi?a Duri? >> > From dragisha at m3w.org Mon Apr 23 10:21:16 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 23 Apr 2007 10:21:16 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> Message-ID: <1177316476.737.97.camel@faramir.m3w.org> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > > > Hello, > > > > Have been skimming through source of PTHREAD code... And I think > > job can > > be done without so much relying on how-they-did-it-before, esp with > > regard to list of waiters and similar internal and global structures. > > Also, I see number of global locks and I am sure they are congestion > > generators every now and while, esp in heavy threading situations. > > > > Of course, there is number of approaches to this multi-thread > > situations. Mine being one of very nonconservative use of threads, I > > think it is important to remain open to possibly very big number of > > threads running in single process - meaning scalability is one of > > primary objections... As global locks don't do well with > > scalability, I > > don't like "cm" and similar global congestion points. > > Yes, there are tensions between a thin/absent veneer between language > threads and system threads. Most important are issues of preserving > a reasonable memory model for programmers (see Hans Boehm's paper > http://portal.acm.org/citation.cfm?doid=1065010.1065042). I know that paper, and being Modula-3 camp, I am - by definition - agreed to no-library-approach-for-threads :). > There are > also questions of portability and debuggability. Of course. That's why I am using only POSIX defined features, and when in doubt - ones used by Boehm in his famous GC :). > I agree that global > locks are to be avoided where they cause known contention, but there > are tradeoffs there too. Global lock is bad, whatever reasons. > For large numbers of threads (as you appear > to need) I think we would need to adopt some other implementation > approach, possibly by multiplexing multiple lightweight user-level > threads on some smaller number of heavyweight system-level threads, > but then you run into scheduling and load-balancing problems. I've argued this before... With O(1) process scheduling available for four years from Linux, and in that time surely from everyone else... It would be bigger problem to maintain scheduling for special "BIG" cases inside our support libraries than to rely on operating system. It's good that mainstream OS people recognized threads as Need, and it is time now for us to accept they did it well - AT LAST. And, very important... I can't see what is heavyweight on system which does 10,000 context switches per second for 1500 threads with 2% CPU load. And all this in 2004 on some 1.something GHz CPU. Threads WERE heavyweight four years ago, and they are not, long since. Even Windows has lightweight kernel-space threads :). > > > We talked about this at least once before, and I think I remember you > > insisted on more compatibility than can be read from SPwM3. Do you > > think > > best idea would be to integrate mine NPTL code into CM3 for people to > > trash and test, and let everyone select what is best for their > > situation? > > What I wanted was a situation where programs would be able to run > with the same tools (e.g., showheap, showthread) under both user- > level and system threading. This goal has been achieved with the > current pthread-based implementation. It is good reasoning, and it's one of reasons I did not suggest replacement... I think mine version is less bloated and I know it's very good for long and stable process uptime we all expect from Modula-3 programs. But also, implied compatibilities outside of SPwM3 and direct demands from other parts of runtime were not on my list. These I've respected, and it looks like these are good production time criteria. As opposed to excellent development time criteria you based yours on. > Moreover, I wanted something > where variations in thread support from one system to another could > be exploited most easily (such as for systems where thread suspend/ > resume is provided as a primitive). Again, the current > implementation achieves this, and runs with minimal target-specific > code on Darwin, Solaris, and Linux. Ports to other targets should be > relatively straightforward. I've not ported mine outside of LINUXLIBC6, but as it's extremmely POSIX, I see no problem. > > > Problems I had with my pthread implementation were all related to VM > > hell of earlier GC implementation... After you did that piece of art > > with new approach to GC, I expect infinite uptimes from my servers and > > bots :). Big thanks for that! > > Any native threading implementation is going to have problems with VM- > based memory management. I'm surprised to were able to get anything > going with the VM-based GC. Anything is pretty much - I have heavy multithreaded servers running literally for years,,, one of them is up since January of 2004, it services few hundreds of connected users (and up to 1500 threads) at almost every moment and breaks only when system reboots :). All that with heavy integration of various C libraries. > > > > > dd > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 14:40:44 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 23 Apr 2007 08:40:44 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177316476.737.97.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: I take all of your points seriously. One option would be to offer your threads implementation as another build option for CM3. I'm going to track down the bug I introduced recently and then we can consider how to move forward. On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >> >>> Hello, >>> >>> Have been skimming through source of PTHREAD code... And I think >>> job can >>> be done without so much relying on how-they-did-it-before, esp with >>> regard to list of waiters and similar internal and global >>> structures. >>> Also, I see number of global locks and I am sure they are congestion >>> generators every now and while, esp in heavy threading situations. >>> >>> Of course, there is number of approaches to this multi-thread >>> situations. Mine being one of very nonconservative use of threads, I >>> think it is important to remain open to possibly very big number of >>> threads running in single process - meaning scalability is one of >>> primary objections... As global locks don't do well with >>> scalability, I >>> don't like "cm" and similar global congestion points. >> >> Yes, there are tensions between a thin/absent veneer between language >> threads and system threads. Most important are issues of preserving >> a reasonable memory model for programmers (see Hans Boehm's paper >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > I know that paper, and being Modula-3 camp, I am - by definition - > agreed to no-library-approach-for-threads :). > >> There are >> also questions of portability and debuggability. > > Of course. That's why I am using only POSIX defined features, and > when > in doubt - ones used by Boehm in his famous GC :). > >> I agree that global >> locks are to be avoided where they cause known contention, but there >> are tradeoffs there too. > > Global lock is bad, whatever reasons. > >> For large numbers of threads (as you appear >> to need) I think we would need to adopt some other implementation >> approach, possibly by multiplexing multiple lightweight user-level >> threads on some smaller number of heavyweight system-level threads, >> but then you run into scheduling and load-balancing problems. > > I've argued this before... With O(1) process scheduling available > for > four years from Linux, and in that time surely from everyone > else... It > would be bigger problem to maintain scheduling for special "BIG" cases > inside our support libraries than to rely on operating system. It's > good > that mainstream OS people recognized threads as Need, and it is > time now > for us to accept they did it well - AT LAST. > > And, very important... I can't see what is heavyweight on system > which > does 10,000 context switches per second for 1500 threads with 2% CPU > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > heavyweight four years ago, and they are not, long since. Even Windows > has lightweight kernel-space threads :). > >> >>> We talked about this at least once before, and I think I remember >>> you >>> insisted on more compatibility than can be read from SPwM3. Do you >>> think >>> best idea would be to integrate mine NPTL code into CM3 for >>> people to >>> trash and test, and let everyone select what is best for their >>> situation? >> >> What I wanted was a situation where programs would be able to run >> with the same tools (e.g., showheap, showthread) under both user- >> level and system threading. This goal has been achieved with the >> current pthread-based implementation. > > It is good reasoning, and it's one of reasons I did not suggest > replacement... I think mine version is less bloated and I know it's > very > good for long and stable process uptime we all expect from Modula-3 > programs. But also, implied compatibilities outside of SPwM3 and > direct > demands from other parts of runtime were not on my list. These I've > respected, and it looks like these are good production time > criteria. As > opposed to excellent development time criteria you based yours on. > >> Moreover, I wanted something >> where variations in thread support from one system to another could >> be exploited most easily (such as for systems where thread suspend/ >> resume is provided as a primitive). Again, the current >> implementation achieves this, and runs with minimal target-specific >> code on Darwin, Solaris, and Linux. Ports to other targets should be >> relatively straightforward. > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > POSIX, I see no problem. > >> >>> Problems I had with my pthread implementation were all related to VM >>> hell of earlier GC implementation... After you did that piece of art >>> with new approach to GC, I expect infinite uptimes from my >>> servers and >>> bots :). Big thanks for that! >> >> Any native threading implementation is going to have problems with >> VM- >> based memory management. I'm surprised to were able to get anything >> going with the VM-based GC. > > Anything is pretty much - I have heavy multithreaded servers running > literally for years,,, one of them is up since January of 2004, it > services few hundreds of connected users (and up to 1500 threads) at > almost every moment and breaks only when system reboots :). All that > with heavy integration of various C libraries. > >> >>> >>> dd >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From mika at async.caltech.edu Tue Apr 24 22:21:25 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 24 Apr 2007 13:21:25 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Tue, 24 Apr 2007 14:15:19 -0000." Message-ID: <200704242021.l3OKLPac036051@camembert.async.caltech.edu> Yes, that is the entire rule. There can definitely be cycles in the graph; in this case, the rule "if M depends on N and N does not depend on M" does not apply, as they depend on each other. The order of initialization is then implementation-dependent. Just creating objects of type (circularly) implemented by another module doesn't present any special problems, unless the correct functioning of those objects is dependent on module-initialization code. And I don't know about your compiler, but mine won't link things that didn't compile. I have to admit that the module initialization rule does break a nice property of Modula-3 programs, which is that you can't normally break a module by making a change to another module. If your module M indirectly depends on module N and furthermore, the correct functioning of the M-N relationship depends on module N's being initialized before module M, the addition in module N (in private implementation code) of a dependency on module M's exported interface can cause your program to fail. Mika "j k" writes: >ok. Can there be circularities in the graph? >Two modules creating objects of types implemented by the other in their >module initializer? Or that won't compile? I'll have to try it. > >- Jay > > >>From: Mika Nystrom >>To: "j k" >>CC: dragisha at m3w.org, m3devel at elegosoft.com >>Subject: Re: [M3devel] order of module initialization... Date: Fri, 20 Apr >>2007 15:15:16 -0700 >> >>The order in Modula-3 is defined in section 2.5.6 of the language >>report (page 47 of Systems Programming with Modula-3). >> >>The problem described here appears to be that EXPORTS isn't treated >>the same as IMPORT (which it should be). There are some other >>problems I noticed when porting PM3 code to CM3, don't know if they >>have been fixed. The "solution" in that case was to IMPORT interfaces >>in places where they weren't really needed. >> >>Obfuscating one's code to avoid taking advantage of the convenience >>of module initialization seems like a rather severe suggestion... >> >>The rule, by the way, is: >> >>"If module M depends on module N and N does not depend on M, then >>N's body will be executed before M's body, where: >> >>* A module M 'depends on' a module N if M uses an interface that N >>exports or if M depends on a module that depends on N. >> >>* A module M 'uses' an interface I if M imports or exports I or if >>M uses an interface that imports I. >> >>Except for this constraint, the order of execution is >>implementation-dependent." >> >> Mika >> >>"j k" writes: >> >Reminds me of C++ globals with constructors. >> > >> >The solution is to avoid initialization. >> > >> >Do everything "on demand" and hide all state behind functions. >> > and then "on demand" needs to be thread safe, so there is a small >> >bootstrapping problem -- basically the only initialization you should do, >> >speaking in Win32-ese, is a few calls to InitializeCriticalSection and >>maybe >> >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >> >accurately, nothing "in the same layer", if you have a dependency graph, >> >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always >>are >> >(unless you are working on ntdll.dll, but even then.. :) ) If you don't >>have >> >a strict layered design, or can't capture what it is and analyze >>statically, >> >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is >>an >> >easy way to be correct. >> > >> >And even then, you can often avoid the critical sections via something >>like >> >InterlockedCompareExchangePointer, even implement spin locks for this >rare >> >code. (Hand written spin locks are generally to be avoided since the >>kernel >> >won't know about them and you won't schedule efficiently, whereas with >> >EnterCriticalSection, if there is contention, the kernel has a hint what >>is >> >going on and can schedule better). >> > >> >Don't have any "public global data", unless, and I don't know if this >> >applies in Modula-3, unless it is "constant initialized", which I'm not >>sure >> >is well defined in C++ or not. I do believe it is well defined in >>C..except >> >dynamic linking I think screws that up to. >> > >> >(pardon the Win32-centricity; I'm sure the principles apply more broadly) >> > >> >Ok, in C++ and in Modula-3, there are maybe better answers. >> >In C++, globals within a source file are guaranteed to be initialized in >>the >> >order they occur in the file. >> >But the order of files is not defined. >> >There is a documented trick..potentially very inefficient..that I don't >>have >> >the patience to describe here.. "the iostream counter trick"... this I >>don't >> >think has an analog in Modula-3, but sure. >> >Stroupstroup's Design and Evolution book documents it. >> > >> >- Jay >> > >> > >> >>From: Dragi??a Duri?? >> >>To: "M3 Developers' Mailing List" >> >>Subject: [M3devel] order of module initialization... >> >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >> >> >>I remember debugging this first time we tried to migrate to CM3... And >> >>then we solved it, for current CM3 and current version of our software, >> >>mentioning some imported module in EXPORTS clause... at the moment, this >> >>linked these modules in initialization chain and it worked ok... Right >> >>now, it does not. >> >> >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >> >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >> >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >> >>wrong moment... >> >> >> >>What would be solution for this? >> >> >> >>dd >> >>-- >> >>Dragi??a Duri?? >> >> >> >>_______________________________________________ >> >>M3devel mailing list >> >>M3devel at elegosoft.com >> >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >_________________________________________________________________ >> >Download Messenger. Join the i?m Initiative. Help make a difference >>today. >> >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 >> > >> >_______________________________________________ >> >M3devel mailing list >> >M3devel at elegosoft.com >> >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Mortgage rates near historic lows. Refinance $200,000 loan for as low as >$771/month* >https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&search=m >ortgage_text_links_88_h27f8&disc=y&vers=689&s=4056&p=5117 From dragisha at m3w.org Wed Apr 25 08:55:12 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 08:55:12 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: <1177484112.4826.9.camel@faramir.m3w.org> Just a random thought... I don't think my TestThreads is something special, but it's few thread use patterns combined... And I've just had bright :) idea yesterday - it's also decent benchmark for whole threading system... I think it would be nice to test it with 10000 rounds, 4000 threads each (once cm3 cvs-head is fixed) with PTHREAD and without PTHREAD. I will do tests for mine. I think these extra data structures and global locks in PTHREAD are big efficiency killers. Benchmark will show. dd On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > I take all of your points seriously. One option would be to offer > your threads implementation as another build option for CM3. I'm > going to track down the bug I introduced recently and then we can > consider how to move forward. > > On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > > > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >> > >>> Hello, > >>> > >>> Have been skimming through source of PTHREAD code... And I think > >>> job can > >>> be done without so much relying on how-they-did-it-before, esp with > >>> regard to list of waiters and similar internal and global > >>> structures. > >>> Also, I see number of global locks and I am sure they are congestion > >>> generators every now and while, esp in heavy threading situations. > >>> > >>> Of course, there is number of approaches to this multi-thread > >>> situations. Mine being one of very nonconservative use of threads, I > >>> think it is important to remain open to possibly very big number of > >>> threads running in single process - meaning scalability is one of > >>> primary objections... As global locks don't do well with > >>> scalability, I > >>> don't like "cm" and similar global congestion points. > >> > >> Yes, there are tensions between a thin/absent veneer between language > >> threads and system threads. Most important are issues of preserving > >> a reasonable memory model for programmers (see Hans Boehm's paper > >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > > > I know that paper, and being Modula-3 camp, I am - by definition - > > agreed to no-library-approach-for-threads :). > > > >> There are > >> also questions of portability and debuggability. > > > > Of course. That's why I am using only POSIX defined features, and > > when > > in doubt - ones used by Boehm in his famous GC :). > > > >> I agree that global > >> locks are to be avoided where they cause known contention, but there > >> are tradeoffs there too. > > > > Global lock is bad, whatever reasons. > > > >> For large numbers of threads (as you appear > >> to need) I think we would need to adopt some other implementation > >> approach, possibly by multiplexing multiple lightweight user-level > >> threads on some smaller number of heavyweight system-level threads, > >> but then you run into scheduling and load-balancing problems. > > > > I've argued this before... With O(1) process scheduling available > > for > > four years from Linux, and in that time surely from everyone > > else... It > > would be bigger problem to maintain scheduling for special "BIG" cases > > inside our support libraries than to rely on operating system. It's > > good > > that mainstream OS people recognized threads as Need, and it is > > time now > > for us to accept they did it well - AT LAST. > > > > And, very important... I can't see what is heavyweight on system > > which > > does 10,000 context switches per second for 1500 threads with 2% CPU > > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > > heavyweight four years ago, and they are not, long since. Even Windows > > has lightweight kernel-space threads :). > > > >> > >>> We talked about this at least once before, and I think I remember > >>> you > >>> insisted on more compatibility than can be read from SPwM3. Do you > >>> think > >>> best idea would be to integrate mine NPTL code into CM3 for > >>> people to > >>> trash and test, and let everyone select what is best for their > >>> situation? > >> > >> What I wanted was a situation where programs would be able to run > >> with the same tools (e.g., showheap, showthread) under both user- > >> level and system threading. This goal has been achieved with the > >> current pthread-based implementation. > > > > It is good reasoning, and it's one of reasons I did not suggest > > replacement... I think mine version is less bloated and I know it's > > very > > good for long and stable process uptime we all expect from Modula-3 > > programs. But also, implied compatibilities outside of SPwM3 and > > direct > > demands from other parts of runtime were not on my list. These I've > > respected, and it looks like these are good production time > > criteria. As > > opposed to excellent development time criteria you based yours on. > > > >> Moreover, I wanted something > >> where variations in thread support from one system to another could > >> be exploited most easily (such as for systems where thread suspend/ > >> resume is provided as a primitive). Again, the current > >> implementation achieves this, and runs with minimal target-specific > >> code on Darwin, Solaris, and Linux. Ports to other targets should be > >> relatively straightforward. > > > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > > POSIX, I see no problem. > > > >> > >>> Problems I had with my pthread implementation were all related to VM > >>> hell of earlier GC implementation... After you did that piece of art > >>> with new approach to GC, I expect infinite uptimes from my > >>> servers and > >>> bots :). Big thanks for that! > >> > >> Any native threading implementation is going to have problems with > >> VM- > >> based memory management. I'm surprised to were able to get anything > >> going with the VM-based GC. > > > > Anything is pretty much - I have heavy multithreaded servers running > > literally for years,,, one of them is up since January of 2004, it > > services few hundreds of connected users (and up to 1500 threads) at > > almost every moment and breaks only when system reboots :). All that > > with heavy integration of various C libraries. > > > >> > >>> > >>> dd > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From wagner at elegosoft.com Wed Apr 25 09:10:20 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:10:20 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: <462EFEDC.5090000@elegosoft.com> Dragi?a Duri? wrote: > I remember debugging this first time we tried to migrate to CM3... And > then we solved it, for current CM3 and current version of our software, > mentioning some imported module in EXPORTS clause... at the moment, this > linked these modules in initialization chain and it worked ok... Right > now, it does not. > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > wrong moment... > > What would be solution for this? > The order of module initialization in CM3 is extensively traced. Try to run your program with @M3tracelinker and see what's wrong. The code for module initialization is in cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 If the initialization does not work as expected, we must be missing of overlooking some dependency there. If you can provide a simple and small example that does not work, I can probably have a look at it at the next weekend. Sorry for the late answer, Olaf From dragisha at m3w.org Wed Apr 25 09:44:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 09:44:52 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <462EFEDC.5090000@elegosoft.com> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> Message-ID: <1177487092.4826.15.camel@faramir.m3w.org> I've debugged this before, and I know the drill... Problem with initialization, sometimes, is in it passing pass 2 of RTLinker for some modules and never coming to pass 3... I've noticed that. Last few days I am using -linkall, and after I've deleted some redundant EXPORTS it kind of works ok. I remember you telling me about this non-initialization of non-imported modules once before. It is where CM3 differs from both SRC and PM3 in big way. Don't remember rationalization, but I don't like situation where I can't make/exploit plugin architecture simply by mentioning new module in m3makefile and letting it initialize itself in standardized manner. dd On Wed, 2007-04-25 at 09:10 +0200, Olaf Wagner wrote: > Dragi?a Duri? wrote: > > I remember debugging this first time we tried to migrate to CM3... And > > then we solved it, for current CM3 and current version of our software, > > mentioning some imported module in EXPORTS clause... at the moment, this > > linked these modules in initialization chain and it worked ok... Right > > now, it does not. > > > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > > wrong moment... > > > > What would be solution for this? > > > The order of module initialization in CM3 is extensively traced. Try to > run your program with @M3tracelinker and see what's wrong. The > code for module initialization is in > > cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 > > If the initialization does not work as expected, we must be missing > of overlooking some dependency there. > > If you can provide a simple and small example that does not work, > I can probably have a look at it at the next weekend. > > Sorry for the late answer, > > Olaf -- Dragi?a Duri? From wagner at mx0.elegosoft.com Wed Apr 25 09:00:22 2007 From: wagner at mx0.elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:00:22 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <20070425070022.GA4827@elegosoft.com> On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I've checked in a fix for this monday evening, thought the commit message again wasn't delivered. I also instructed Ronny to have a look at the commit message setup again. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > > >while trying to make my programs with newsly installed cm3, I am > >constantly running into same error. Example: > > > >import("libm3") > > > >include_dir("../../../../m3lib0/src") > >include_dir("../../../../pdf/src") > >include_dir("../../../../zip/src") > > > >implementation("Test") > >program("test") > > > >Is my m3makefile, and when "cm3" is run in package's src dir, I have > >only: > > > >faramir:dragisha/pts/9: test/1/src% cm3 > >--- building in ../LINUXLIBC6 --- > > > > > > > >*** > >*** runtime error: > >*** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > >list > >*** file "../src/os/POSIX/PathnamePosix.m3", line 98 > >*** > > > >Mentioned line raises this FATAL exception when Absolute(path) is > >TRUE... What is absolute in my paths? > > > >All m3makefiles in question were ok with pm3, of course.. > > > >dd > >-- > >Dragi??a Duri?? > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Wed Apr 25 11:33:44 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 11:33:44 +0200 (CEST) Subject: [M3devel] order of module initialization... In-Reply-To: <1177487092.4826.15.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> <1177487092.4826.15.camel@faramir.m3w.org> Message-ID: <62295.194.138.127.36.1177493624.squirrel@mail.elegosoft.com> On Wed, April 25, 2007 9:44 am, Dragi??a Duri?� wrote: > I've debugged this before, and I know the drill... Problem with > initialization, sometimes, is in it passing pass 2 of RTLinker for some > modules and never coming to pass 3... I've noticed that. Last few days I > am using -linkall, and after I've deleted some redundant EXPORTS it kind > of works ok. > > I remember you telling me about this non-initialization of non-imported > modules once before. It is where CM3 differs from both SRC and PM3 in > big way. Don't remember rationalization, but I don't like situation > where I can't make/exploit plugin architecture simply by mentioning new > module in m3makefile and letting it initialize itself in standardized > manner. I'm afraid I still don't understand the problem well enough though I think we've probably got a bug somewhere in the initialization code. It would be really helpful if you could provide a simple example for debugging. I'll take care of the problem then (though it may take some time). Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 25 15:51:52 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:51:52 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177484112.4826.9.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: Yes, good thinking. Tuning the threads systems is a good plan all round. On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > Just a random thought... > > I don't think my TestThreads is something special, but it's few thread > use patterns combined... And I've just had bright :) idea yesterday - > it's also decent benchmark for whole threading system... I think it > would be nice to test it with 10000 rounds, 4000 threads each (once > cm3 > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > tests for > mine. > > I think these extra data structures and global locks in PTHREAD are > big > efficiency killers. Benchmark will show. > > dd > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >> I take all of your points seriously. One option would be to offer >> your threads implementation as another build option for CM3. I'm >> going to track down the bug I introduced recently and then we can >> consider how to move forward. >> >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >> >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>> >>>>> Hello, >>>>> >>>>> Have been skimming through source of PTHREAD code... And I think >>>>> job can >>>>> be done without so much relying on how-they-did-it-before, esp >>>>> with >>>>> regard to list of waiters and similar internal and global >>>>> structures. >>>>> Also, I see number of global locks and I am sure they are >>>>> congestion >>>>> generators every now and while, esp in heavy threading situations. >>>>> >>>>> Of course, there is number of approaches to this multi-thread >>>>> situations. Mine being one of very nonconservative use of >>>>> threads, I >>>>> think it is important to remain open to possibly very big >>>>> number of >>>>> threads running in single process - meaning scalability is one of >>>>> primary objections... As global locks don't do well with >>>>> scalability, I >>>>> don't like "cm" and similar global congestion points. >>>> >>>> Yes, there are tensions between a thin/absent veneer between >>>> language >>>> threads and system threads. Most important are issues of >>>> preserving >>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>> >>> I know that paper, and being Modula-3 camp, I am - by definition - >>> agreed to no-library-approach-for-threads :). >>> >>>> There are >>>> also questions of portability and debuggability. >>> >>> Of course. That's why I am using only POSIX defined features, and >>> when >>> in doubt - ones used by Boehm in his famous GC :). >>> >>>> I agree that global >>>> locks are to be avoided where they cause known contention, but >>>> there >>>> are tradeoffs there too. >>> >>> Global lock is bad, whatever reasons. >>> >>>> For large numbers of threads (as you appear >>>> to need) I think we would need to adopt some other implementation >>>> approach, possibly by multiplexing multiple lightweight user-level >>>> threads on some smaller number of heavyweight system-level threads, >>>> but then you run into scheduling and load-balancing problems. >>> >>> I've argued this before... With O(1) process scheduling available >>> for >>> four years from Linux, and in that time surely from everyone >>> else... It >>> would be bigger problem to maintain scheduling for special "BIG" >>> cases >>> inside our support libraries than to rely on operating system. It's >>> good >>> that mainstream OS people recognized threads as Need, and it is >>> time now >>> for us to accept they did it well - AT LAST. >>> >>> And, very important... I can't see what is heavyweight on system >>> which >>> does 10,000 context switches per second for 1500 threads with 2% CPU >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE >>> heavyweight four years ago, and they are not, long since. Even >>> Windows >>> has lightweight kernel-space threads :). >>> >>>> >>>>> We talked about this at least once before, and I think I remember >>>>> you >>>>> insisted on more compatibility than can be read from SPwM3. Do you >>>>> think >>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>> people to >>>>> trash and test, and let everyone select what is best for their >>>>> situation? >>>> >>>> What I wanted was a situation where programs would be able to run >>>> with the same tools (e.g., showheap, showthread) under both user- >>>> level and system threading. This goal has been achieved with the >>>> current pthread-based implementation. >>> >>> It is good reasoning, and it's one of reasons I did not suggest >>> replacement... I think mine version is less bloated and I know it's >>> very >>> good for long and stable process uptime we all expect from Modula-3 >>> programs. But also, implied compatibilities outside of SPwM3 and >>> direct >>> demands from other parts of runtime were not on my list. These I've >>> respected, and it looks like these are good production time >>> criteria. As >>> opposed to excellent development time criteria you based yours on. >>> >>>> Moreover, I wanted something >>>> where variations in thread support from one system to another could >>>> be exploited most easily (such as for systems where thread suspend/ >>>> resume is provided as a primitive). Again, the current >>>> implementation achieves this, and runs with minimal target-specific >>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>> should be >>>> relatively straightforward. >>> >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely >>> POSIX, I see no problem. >>> >>>> >>>>> Problems I had with my pthread implementation were all related >>>>> to VM >>>>> hell of earlier GC implementation... After you did that piece >>>>> of art >>>>> with new approach to GC, I expect infinite uptimes from my >>>>> servers and >>>>> bots :). Big thanks for that! >>>> >>>> Any native threading implementation is going to have problems with >>>> VM- >>>> based memory management. I'm surprised to were able to get >>>> anything >>>> going with the VM-based GC. >>> >>> Anything is pretty much - I have heavy multithreaded servers >>> running >>> literally for years,,, one of them is up since January of 2004, it >>> services few hundreds of connected users (and up to 1500 threads) at >>> almost every moment and breaks only when system reboots :). All that >>> with heavy integration of various C libraries. >>> >>>> >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed Apr 25 15:52:49 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:52:49 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <20070425070022.GA4827@elegosoft.com> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> <20070425070022.GA4827@elegosoft.com> Message-ID: PS CVS head is broken right now due to my most recent "fixes" to threading. I am currently looking into the problem. On Apr 25, 2007, at 3:00 AM, Olaf Wagner wrote: > On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: >> I have my hands full with GC evolution and thread issues at the >> moment. Anyone else care to look into this? > > I've checked in a fix for this monday evening, thought the commit > message again wasn't delivered. I also instructed Ronny to have > a look at the commit message setup again. > > Olaf > >> On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: >> >>> while trying to make my programs with newsly installed cm3, I am >>> constantly running into same error. Example: >>> >>> import("libm3") >>> >>> include_dir("../../../../m3lib0/src") >>> include_dir("../../../../pdf/src") >>> include_dir("../../../../zip/src") >>> >>> implementation("Test") >>> program("test") >>> >>> Is my m3makefile, and when "cm3" is run in package's src dir, I have >>> only: >>> >>> faramir:dragisha/pts/9: test/1/src% cm3 >>> --- building in ../LINUXLIBC6 --- >>> >>> >>> >>> *** >>> *** runtime error: >>> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >>> list >>> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >>> *** >>> >>> Mentioned line raises this FATAL exception when Absolute(path) is >>> TRUE... What is absolute in my paths? >>> >>> All m3makefiles in question were ok with pm3, of course.. >>> >>> dd >>> -- >>> Dragi??a Duri?? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Wed Apr 25 18:35:37 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 18:35:37 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: <1177518937.15060.6.camel@faramir.m3w.org> After implementing that workaround for result code 11 in that SIGSUSPEND loop, every time during 1st or at most second pass (of 4000) it stucks. Not same place every time, though... I think there are RCs in LockHeap/SuspendAll logic. dd On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > Yes, good thinking. Tuning the threads systems is a good plan all > round. > > On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > > > Just a random thought... > > > > I don't think my TestThreads is something special, but it's few thread > > use patterns combined... And I've just had bright :) idea yesterday - > > it's also decent benchmark for whole threading system... I think it > > would be nice to test it with 10000 rounds, 4000 threads each (once > > cm3 > > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > > tests for > > mine. > > > > I think these extra data structures and global locks in PTHREAD are > > big > > efficiency killers. Benchmark will show. > > > > dd > > > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >> I take all of your points seriously. One option would be to offer > >> your threads implementation as another build option for CM3. I'm > >> going to track down the bug I introduced recently and then we can > >> consider how to move forward. > >> > >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >> > >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>> > >>>>> Hello, > >>>>> > >>>>> Have been skimming through source of PTHREAD code... And I think > >>>>> job can > >>>>> be done without so much relying on how-they-did-it-before, esp > >>>>> with > >>>>> regard to list of waiters and similar internal and global > >>>>> structures. > >>>>> Also, I see number of global locks and I am sure they are > >>>>> congestion > >>>>> generators every now and while, esp in heavy threading situations. > >>>>> > >>>>> Of course, there is number of approaches to this multi-thread > >>>>> situations. Mine being one of very nonconservative use of > >>>>> threads, I > >>>>> think it is important to remain open to possibly very big > >>>>> number of > >>>>> threads running in single process - meaning scalability is one of > >>>>> primary objections... As global locks don't do well with > >>>>> scalability, I > >>>>> don't like "cm" and similar global congestion points. > >>>> > >>>> Yes, there are tensions between a thin/absent veneer between > >>>> language > >>>> threads and system threads. Most important are issues of > >>>> preserving > >>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>> > >>> I know that paper, and being Modula-3 camp, I am - by definition - > >>> agreed to no-library-approach-for-threads :). > >>> > >>>> There are > >>>> also questions of portability and debuggability. > >>> > >>> Of course. That's why I am using only POSIX defined features, and > >>> when > >>> in doubt - ones used by Boehm in his famous GC :). > >>> > >>>> I agree that global > >>>> locks are to be avoided where they cause known contention, but > >>>> there > >>>> are tradeoffs there too. > >>> > >>> Global lock is bad, whatever reasons. > >>> > >>>> For large numbers of threads (as you appear > >>>> to need) I think we would need to adopt some other implementation > >>>> approach, possibly by multiplexing multiple lightweight user-level > >>>> threads on some smaller number of heavyweight system-level threads, > >>>> but then you run into scheduling and load-balancing problems. > >>> > >>> I've argued this before... With O(1) process scheduling available > >>> for > >>> four years from Linux, and in that time surely from everyone > >>> else... It > >>> would be bigger problem to maintain scheduling for special "BIG" > >>> cases > >>> inside our support libraries than to rely on operating system. It's > >>> good > >>> that mainstream OS people recognized threads as Need, and it is > >>> time now > >>> for us to accept they did it well - AT LAST. > >>> > >>> And, very important... I can't see what is heavyweight on system > >>> which > >>> does 10,000 context switches per second for 1500 threads with 2% CPU > >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > >>> heavyweight four years ago, and they are not, long since. Even > >>> Windows > >>> has lightweight kernel-space threads :). > >>> > >>>> > >>>>> We talked about this at least once before, and I think I remember > >>>>> you > >>>>> insisted on more compatibility than can be read from SPwM3. Do you > >>>>> think > >>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>> people to > >>>>> trash and test, and let everyone select what is best for their > >>>>> situation? > >>>> > >>>> What I wanted was a situation where programs would be able to run > >>>> with the same tools (e.g., showheap, showthread) under both user- > >>>> level and system threading. This goal has been achieved with the > >>>> current pthread-based implementation. > >>> > >>> It is good reasoning, and it's one of reasons I did not suggest > >>> replacement... I think mine version is less bloated and I know it's > >>> very > >>> good for long and stable process uptime we all expect from Modula-3 > >>> programs. But also, implied compatibilities outside of SPwM3 and > >>> direct > >>> demands from other parts of runtime were not on my list. These I've > >>> respected, and it looks like these are good production time > >>> criteria. As > >>> opposed to excellent development time criteria you based yours on. > >>> > >>>> Moreover, I wanted something > >>>> where variations in thread support from one system to another could > >>>> be exploited most easily (such as for systems where thread suspend/ > >>>> resume is provided as a primitive). Again, the current > >>>> implementation achieves this, and runs with minimal target-specific > >>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>> should be > >>>> relatively straightforward. > >>> > >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely > >>> POSIX, I see no problem. > >>> > >>>> > >>>>> Problems I had with my pthread implementation were all related > >>>>> to VM > >>>>> hell of earlier GC implementation... After you did that piece > >>>>> of art > >>>>> with new approach to GC, I expect infinite uptimes from my > >>>>> servers and > >>>>> bots :). Big thanks for that! > >>>> > >>>> Any native threading implementation is going to have problems with > >>>> VM- > >>>> based memory management. I'm surprised to were able to get > >>>> anything > >>>> going with the VM-based GC. > >>> > >>> Anything is pretty much - I have heavy multithreaded servers > >>> running > >>> literally for years,,, one of them is up since January of 2004, it > >>> services few hundreds of connected users (and up to 1500 threads) at > >>> almost every moment and breaks only when system reboots :). All that > >>> with heavy integration of various C libraries. > >>> > >>>> > >>>>> > >>>>> dd > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From darko at darko.org Tue Apr 3 15:33:23 2007 From: darko at darko.org (Darko) Date: Tue, 3 Apr 2007 15:33:23 +0200 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> Message-ID: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> I'm not sure if this has been resolved in some more proper way, but for the record, the warnings appearing after installation of the XCode 2.4.1 tools can be gotten rid of by changing the following in the config file: proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-o", object) end so it reads proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-W -o", object) end It's pretty obvious, but just in case anyone goes searching these archives for an answer... On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: > >> >> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >> >>> For some reason your .stabs entries are different than mine, but >>> otherwise the assembler files are the same. Are you using the >>> cm3cg and cm3.cfg from my ftp site? >> >> Yes, downloaded a few hours ago. >> >> >>> Here is the version of the assembler that I have on my Intel Mac: >>> >>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>> assembler version 1.38 >> >> The mine is: >> >> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >> version 1.38 >> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >> > > Hmmm -- different versions of the assembler. Probably some flag > needs adjusting. > >> >> >>> >>> With respect to zeus, can you build with -verbose in the zeus >>> directory and send me the output? Sounds like one of the tools >>> is not getting built properly. >>> >> >> Ok, I am sending it as attachment >> >> >> > > Sorry, wrong option. Please run with -commands. I suspect stubgen > is failing for you. > > >> Renzo >> >> >>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>> >>>> You can find in the attachment the program and the compilation. >>>> >>>> Renzo >>>> >>>> >>>> >>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>> >>>>> >>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>> >>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>> >>>>>>> >>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>> >>>>>>>> Hello and lot of thanks! >>>>>>>> >>>>>>>> Everything is working really fine! >>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>> anonymous CVS), >>>>>>>> The process: >>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>> 3. running as su: >>>>>>>> do-cm3-core.sh buildship >>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>> do-cm3-std.sh buildship >>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>> and failing at the end the compilation of package zeus >>>>>>>> (which I suppose is an old issue) >>>>>>> >>>>>>> That's odd since my builds go through cleanly. >>>>>> >>>>>> This is an example of the strange "indirect jump" message: >>>>>> >>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>> --- building in I386_DARWIN --- >>>>>> new source -> compiling Main.m3 >>>>>> Main.ms:101:indirect jmp without `*' >>>>>> -> linking prog >>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>> Hello World >>>>>> >>>>>> maybe there is some problem with the linker parameters, >>>>>> however the progrum runs ok. >>>>> >>>>> Can you compile with "-keep" and send me the relevant .ms >>>>> file? I can compare with mine. >>>>> >>>>>> >>>>>> For what concerns zeus: the following is the output of cm3: >>>>>> >>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>> cvsstuff/cm3' +++ >>>>>> --- building in I386_DARWIN --- >>>>>> >>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> compilation failed => not building library "libm3zeus.a" >>>>>> Fatal Error: package build failed >>>>>> *** execution of failed *** >>>>>> >>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>> >>>>>> Several other packages cannot be compiled, like many obliq >>>>>> ones, webvt, June and mentor. >>>>>> >>>>>> However the compiler seems to work for compiling my stuff, >>>>>> altough I have to make a few corrections and debugging. >>>>> >>>>> This sounds problematic since it implies that the generators >>>>> for those files are not working correctly. (If you run with "- >>>>> commands" you'll see which generator programs are being used. >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Renzo >>>>>> >>>>>>> >>>>>>>> >>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>> century! >>>>>>>> >>>>>>>> Cordially, >>>>>>>> >>>>>>>> Renzo Orsini >>>>>>>> >>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>> >>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>> about 1G, by the way. >>>>>>>>> >>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>> >>>>>>>>>> What version of OSX are you using? >>>>>>>>>> >>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>> >>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>> >>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>> >>>>>>>>>>> - Darko >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>> >>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>> >>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/local/ >>>>>>>>>>>> cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>> this is the result: >>>>>>>>>>>> >>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>> CM3C = >>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh -c >>>>>>>>>>>> "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector >>>>>>>>>>>> digraph parseparams realgeometry set slisp >>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: pointer >>>>>>>>>>>> targets in passing argument 3 of 'accept' differ in >>>>>>>>>>>> signedness >>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>> errno=2 >>>>>>>>>>>> >>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>> install_file -- >>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>> >>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>> >>>>>>>>>>>> ====== >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I do >>>>>>>>>>>> not know if this is a normal behaviour. >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>> >>>>>>>>>>>> Cordially >>>>>>>>>>>> >>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>> >>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>> >>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>> >>>>>>>>>>>>> Darko. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo Orsini >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on an >>>>>>>>>>>>>>> intel mac, then >>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Since I would like to port the Fibonacci language for >>>>>>>>>>>>>>> databases, >>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>> documentation to start with. Of course I will be very >>>>>>>>>>>>>>> glad to give >>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>> >>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>> to install >>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>> be too >>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>> on it, >>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>> mail to >>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>> such an >>>>>>>>>>>>>> effort. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>> m3devel >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> M3devel mailing list >>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>> >>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>> ~hosking >>>>>>>>>> _--_|\ >>>>>>>>>> / \ >>>>>>>>>> \_.--._/ ) >>>>>>>>>> v / >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Antony Hosking | Associate Professor >>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>> ~hosking >>>>>>> _--_|\ >>>>>>> / \ >>>>>>> \_.--._/ ) >>>>>>> v / >>>>>>> >>>>>> >>>>> >>>>> Antony Hosking | Associate Professor >>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>> Purdue University | Mobile: (765) 427-5484 >>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>> _--_|\ >>>>> / \ >>>>> \_.--._/ ) >>>>> v / >>>>> >>>> >>> >>> Antony Hosking | Associate Professor >>> Dept of Computer Sciences | Office: (765) 494-6001 >>> Purdue University | Mobile: (765) 427-5484 >>> 250 N. University Street | hosking at cs.purdue.edu >>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>> _--_|\ >>> / \ >>> \_.--._/ ) >>> v / >>> >> > > Antony Hosking | Associate Professor > Dept of Computer Sciences | Office: (765) 494-6001 > Purdue University | Mobile: (765) 427-5484 > 250 N. University Street | hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Tue Apr 3 16:31:13 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 3 Apr 2007 10:31:13 -0400 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> Message-ID: <9753494C-96FE-4BDE-81AE-8CDDE9BC398B@cs.purdue.edu> -q also works. On Apr 3, 2007, at 9:33 AM, Darko wrote: > I'm not sure if this has been resolved in some more proper way, but > for the record, the warnings appearing after installation of the > XCode 2.4.1 tools can be gotten rid of by changing the following in > the config file: > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-o", object) > end > > so it reads > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-W -o", object) > end > > It's pretty obvious, but just in case anyone goes searching these > archives for an answer... > > > On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > >> On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: >> >>> >>> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >>> >>>> For some reason your .stabs entries are different than mine, but >>>> otherwise the assembler files are the same. Are you using the >>>> cm3cg and cm3.cfg from my ftp site? >>> >>> Yes, downloaded a few hours ago. >>> >>> >>>> Here is the version of the assembler that I have on my Intel Mac: >>>> >>>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>>> assembler version 1.38 >>> >>> The mine is: >>> >>> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >>> version 1.38 >>> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >>> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >>> >> >> Hmmm -- different versions of the assembler. Probably some flag >> needs adjusting. >> >>> >>> >>>> >>>> With respect to zeus, can you build with -verbose in the zeus >>>> directory and send me the output? Sounds like one of the tools >>>> is not getting built properly. >>>> >>> >>> Ok, I am sending it as attachment >>> >>> >>> >> >> Sorry, wrong option. Please run with -commands. I suspect >> stubgen is failing for you. >> >> >>> Renzo >>> >>> >>>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>>> >>>>> You can find in the attachment the program and the compilation. >>>>> >>>>> Renzo >>>>> >>>>> >>>>> >>>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>>> >>>>>> >>>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>>> >>>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>>> >>>>>>>> >>>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>>> >>>>>>>>> Hello and lot of thanks! >>>>>>>>> >>>>>>>>> Everything is working really fine! >>>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>>> anonymous CVS), >>>>>>>>> The process: >>>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>>> 3. running as su: >>>>>>>>> do-cm3-core.sh buildship >>>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>>> do-cm3-std.sh buildship >>>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>>> and failing at the end the compilation of package zeus >>>>>>>>> (which I suppose is an old issue) >>>>>>>> >>>>>>>> That's odd since my builds go through cleanly. >>>>>>> >>>>>>> This is an example of the strange "indirect jump" message: >>>>>>> >>>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>> --- building in I386_DARWIN --- >>>>>>> new source -> compiling Main.m3 >>>>>>> Main.ms:101:indirect jmp without `*' >>>>>>> -> linking prog >>>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>>> Hello World >>>>>>> >>>>>>> maybe there is some problem with the linker parameters, >>>>>>> however the progrum runs ok. >>>>>> >>>>>> Can you compile with "-keep" and send me the relevant .ms >>>>>> file? I can compare with mine. >>>>>> >>>>>>> >>>>>>> For what concerns zeus: the following is the output of cm3: >>>>>>> >>>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>>> cvsstuff/cm3' +++ >>>>>>> --- building in I386_DARWIN --- >>>>>>> >>>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> compilation failed => not building library "libm3zeus.a" >>>>>>> Fatal Error: package build failed >>>>>>> *** execution of failed *** >>>>>>> >>>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>>> >>>>>>> Several other packages cannot be compiled, like many obliq >>>>>>> ones, webvt, June and mentor. >>>>>>> >>>>>>> However the compiler seems to work for compiling my stuff, >>>>>>> altough I have to make a few corrections and debugging. >>>>>> >>>>>> This sounds problematic since it implies that the generators >>>>>> for those files are not working correctly. (If you run with "- >>>>>> commands" you'll see which generator programs are being used. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Renzo >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>>> century! >>>>>>>>> >>>>>>>>> Cordially, >>>>>>>>> >>>>>>>>> Renzo Orsini >>>>>>>>> >>>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>>> >>>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>>> about 1G, by the way. >>>>>>>>>> >>>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>>> >>>>>>>>>>> What version of OSX are you using? >>>>>>>>>>> >>>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>>> >>>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>>> >>>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>>> >>>>>>>>>>>> - Darko >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>>> >>>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>>> >>>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/ >>>>>>>>>>>>> local/cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>>> this is the result: >>>>>>>>>>>>> >>>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>>> CM3C = >>>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh - >>>>>>>>>>>>> c "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle >>>>>>>>>>>>> bitvector digraph parseparams realgeometry set slisp >>>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'accept' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>>> errno=2 >>>>>>>>>>>>> >>>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>>> install_file -- >>>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>>> >>>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>>> >>>>>>>>>>>>> ====== >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I >>>>>>>>>>>>> do not know if this is a normal behaviour. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>>> >>>>>>>>>>>>> Cordially >>>>>>>>>>>>> >>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Darko. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo >>>>>>>>>>>>>>> Orsini wrote: >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on >>>>>>>>>>>>>>>> an intel mac, then >>>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Since I would like to port the Fibonacci language >>>>>>>>>>>>>>>> for databases, >>>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>>> documentation to start with. Of course I will be >>>>>>>>>>>>>>>> very glad to give >>>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>>> to install >>>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>>> be too >>>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>>> on it, >>>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>>> mail to >>>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>>> such an >>>>>>>>>>>>>>> effort. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>>> m3devel >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>>> >>>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>>> ~hosking >>>>>>>>>>> _--_|\ >>>>>>>>>>> / \ >>>>>>>>>>> \_.--._/ ) >>>>>>>>>>> v / >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Antony Hosking | Associate Professor >>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>> ~hosking >>>>>>>> _--_|\ >>>>>>>> / \ >>>>>>>> \_.--._/ ) >>>>>>>> v / >>>>>>>> >>>>>>> >>>>>> >>>>>> Antony Hosking | Associate Professor >>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>>> _--_|\ >>>>>> / \ >>>>>> \_.--._/ ) >>>>>> v / >>>>>> >>>>> >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>> Purdue University | Mobile: (765) 427-5484 >>>> 250 N. University Street | hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>> >> >> Antony Hosking | Associate Professor >> Dept of Computer Sciences | Office: (765) 494-6001 >> Purdue University | Mobile: (765) 427-5484 >> 250 N. University Street | hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From stsp at elego.de Mon Apr 9 14:21:31 2007 From: stsp at elego.de (Stefan Sperling) Date: Mon, 9 Apr 2007 14:21:31 +0200 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461A2D05.6040208@elego.de> References: <46192AF7.6060406@elego.de> <20070408194650.GB32688@jack.stsp.lan> <4619836C.6060707@elego.de> <461A2D05.6040208@elego.de> Message-ID: <20070409122131.GB1350@ted.stsp.lan> On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels Janosch Hofmeyr wrote: > Just to complete the information in this thread: > ubuntu 6.10 does not have an asm/ipc.h. However, asm/ipc.h is not > necessary. It is sufficient to delete or comment the lines 19-21 in file > m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > #include > #endif > > This change has been committed to the repository, but was accidentally > not merged into the 5.4.0 release, apparently. Shall I create new source archives for the 5.4 branch that include this fix? If so, what version number should I put on them? 5.4.1? It should not be necessary to update the binary packages as well. Should I just copy them to a new name with the updated version number? -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From dabenavidesd at yahoo.es Mon Apr 9 21:47:35 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 9 Apr 2007 21:47:35 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409122131.GB1350@ted.stsp.lan> Message-ID: <636229.78700.qm@web27107.mail.ukl.yahoo.com> --- Stefan Sperling wrote: >On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels >Janosch Hofmeyr wrote: > Just to complete the information in this thread: > >It is sufficient to delete or comment > >the lines 19-21 in file > > > >m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > > #include > > #endif > > > > This change has been committed to the repository, > but was accidentally > > not merged into the 5.4.0 release, apparently. The problem was reported on the m3devel list on february 6, so It couldnt be merged on cm3-src-all-5.4.0.tgz file if that is the proeblem, that you refer. > Shall I create new source archives for the 5.4 > branch that include this > fix? If so, what version number should I put on > them? 5.4.1? I think this is the most appropiate to put a new file to download with the sources updated (not just that sysdeps.c update). > It should not be necessary to update the binary > packages as well. > Should I just copy them to a new name with the > updated version number? Antony, the bootstrap for the Pthread version is different from the bootstrap of cm3-5.4.0 release? I have exectuted do-cm3-std.sh buildship with the current cvs sources with cm3-5.4.0 bootstrap and in just one package I get a problem; Juno-2 with ubuntu 6.10, when tries to do a redirection of the of the PklFonts executable. ./PklFonts > FontData.pkl It got segmentation fault The thing is, after the rest of the system is compiled the Juno-2 compiles with no problems, including that redirection. ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From dabenavidesd at yahoo.es Tue Apr 10 01:49:25 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 01:49:25 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: Message-ID: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Hello: I have updated the libfreetype6 (to libfreetype6 2.2.1-5) library in ubuntu 6.10, and its now corrected the problem on juno with PklFonts, because, I have executed do-cm3-std.sh buildship with all def-std-pkg.sh packages and there was no problem when doing ./PklFonts > FontData.pkl However the Juno executable does not run, but this error shows: admin07 at sl07:~$ Juno *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "../src/runtime/common/RTCollector.m3", line 361 *** Cancelado admin07 at sl07:~$ Thanks --- "Daniel Alejandro Benavides D." wrote: > > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From hosking at cs.purdue.edu Tue Apr 10 02:15:59 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 9 Apr 2007 20:15:59 -0400 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> References: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Message-ID: <9ED98160-9379-4A83-951B-F07959ECDBEB@cs.purdue.edu> This looks like something wrong with your build. Please run "do-cm3- std.sh realclean" before "buildship". On Apr 9, 2007, at 7:49 PM, Daniel Alejandro Benavides D. wrote: > Hello: > I have updated the libfreetype6 (to libfreetype6 > 2.2.1-5) library in ubuntu 6.10, and its now corrected > the problem on juno with PklFonts, because, I have > executed do-cm3-std.sh buildship with all > def-std-pkg.sh packages and there was no problem when > doing ./PklFonts > FontData.pkl > > However the Juno executable does not run, but this > error shows: > > admin07 at sl07:~$ Juno > > > *** > *** runtime error: > *** An enumeration or subrange value was out of > range. > *** file "../src/runtime/common/RTCollector.m3", > line 361 > *** > > Cancelado > admin07 at sl07:~$ > > > Thanks > > --- "Daniel Alejandro Benavides D." > wrote: > >> >> >> I have exectuted do-cm3-std.sh buildship with the >> current cvs sources with cm3-5.4.0 bootstrap and in >> just one package I get a problem; Juno-2 with >> ubuntu >> 6.10, when tries to do a redirection of the of the >> PklFonts executable. >> ./PklFonts > FontData.pkl >> >> It got segmentation fault >> >> The thing is, after the rest of the system is >> compiled >> the Juno-2 compiles with no problems, including that >> redirection. >> >> > > > > ______________________________________________ > LLama Gratis a cualquier PC del Mundo. > Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. > http://es.voice.yahoo.com > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Tue Apr 10 04:30:46 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 04:30:46 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461AED47.8090906@elego.de> Message-ID: <20070410023046.3362.qmail@web27103.mail.ukl.yahoo.com> Hi: Can you check the version of libfreetype6? Mus be 2.2.1-5. The other you can do to check Juno-2 is cd to the path of juno and build it manually with cm3 -ship, in the right order of the directories: pkl-fonts juno-machine juno-compiler juno-app > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. Mentor does not depend on juno-2. What is the problem compiling mentor? Thanks --- Neels Janosch Hofmeyr wrote: > having gotten this useful information: > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > > I pondered a bit to find the right way of un-listing > juno-2. In the end > it turned out that I can comment lines 149 to 152 > from > scripts/def-std-pkgs.sh: > > # The Juno-2 graphical constraint based editor > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} pkl-fonts" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-machine" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-compiler" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-app" > > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. > > However, undoing the comments and compiling again > did not succeed. I > guess I'll have to live without the Juno-2 graphical > constraint based > editor... unless you have another suggestion, that > is. > > thanks, > neels > > Neels Janosch Hofmeyr wrote: > > Sorry, no success here... Do I really need this > juno thing? > > > > [...] > > === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > > +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > > -DROOT='/home/neels/elego/tmp/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > > Segmentation fault (core dumped) > > > "/home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile", > line > > 24: quake runtime error: exit 139: cd > ../pkl-fonts/LINUXLIBC6 && > > ./PklFonts > FontData.pkl > > > > --procedure-- -line- -file--- > > exec -- > > include_dir 24 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile > > 5 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/LINUXLIBC6/m3make.args > > > > Fatal Error: package build failed > > *** execution of failed *** > > > > $ env | grep LD > > LD_POINTER_GUARD=0 > > > > > > Stefan Sperling wrote: > > > >> On Mon, Apr 09, 2007 at 02:38:34PM +0200, Neels > Janosch Hofmeyr wrote: > >> > >> > >>> More problems have shown up with cm3 5.4.0 on my > ubuntu 6.10: > >>> > >>> " > >>> === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > >>> +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > >>> -DROOT='/home/neels/elego/tmp/cm3' +++ > >>> --- building in LINUXLIBC6 --- > >>> > >>> ignoring ../src/m3overrides > >>> > >>> cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > >>> Segmentation fault (core dumped) > >>> > >>> > >> Try this (from known problems page): > >> > >> Platforms: Fedora Core Linux, maybe other > distributions > >> > >> Description: All Modula3 programs (including cm3 > itself) exit with a > >> segmentation violation. > >> > >> Solution: Set the LD_POINTER_GUARD environment > variable to 0 before > >> running Modula3 programs, using a command like > "export > >> LD_POINTER_GUARD=0" or equivalent. > >> > >> > >> > > > > > > -- > Neels Janosch Hofmeyr > Software Developer > > neels at elego.de > Public Key: > http://binarchy.net/neels/neels.hofmeyr.public.key.asc > > elego Software Solutions GmbH > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > > ____________________________________________________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From darko at darko.org Tue Apr 10 21:01:54 2007 From: darko at darko.org (Darko) Date: Tue, 10 Apr 2007 21:01:54 +0200 Subject: [M3devel] Native M3 Implementation on Symbian Phones Message-ID: <07E52AF0-4E02-4AB5-88AE-41E4C1BC7F75@darko.org> It's interesting to note that Nokia have released 'OpenC' libraries < http://forum.nokia.com/openc > which seem to provide Posix like runtime and library support. There are limitations which may cause problems with the standard M3 runtime, but its certainly a step closer to being able to target such devices. Darko. From darko at darko.org Wed Apr 18 03:51:27 2007 From: darko at darko.org (Darko) Date: Wed, 18 Apr 2007 03:51:27 +0200 Subject: [M3devel] RTHooks CheckStoreTraced and CheckLoadTraced Message-ID: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Hi, Wondering if you can explain the use of these calls a little more. I'm currently using type maps to read and write fields from traced objects. Reading a traced reference from inside a traced object into a local variable is not working as it should. Should I use CheckLoadTraced and if so when and how? Looking at your changes to RTTypeMap, writing references into objects means you need to call CheckStoreTraced on the object written inside of, before it is written? Cheers, Darko. From wagner at plane.elego.de Wed Apr 18 08:04:18 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Wed, 18 Apr 2007 08:04:18 +0200 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] Message-ID: <20070418060417.GB14308@elegosoft.com> Is this the stack encryption problem again? Has anybody tried on Suse 10.1? Olaf ----- Forwarded message from Pino Zollo ----- DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= From: Pino Zollo To: m3-support at elego.de Subject: Installation Problem Date: Tue, 17 Apr 2007 18:00:50 -0400 X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no version=3.0.3 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Dear Sirs, I have installed the last version of cm3 but I have the following problem: cm3 compiles all programs, but are correctly executed only the ones compiled with the option build_standalone(). The programs which use shared libraries end in segmentation fault. Also the script do-cm3-std.sh fails to compile everything for the same problem: the program PklFonts in the directory m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault because needs shared libraries. Any suggestion ? I am using the distribution SuSE 10.1 with kernel 2.4.27 In the installation I have found that the command "export" was not doing properly its job, so I had to put the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the path in /etc/ld.so.conf I noticed that executables contain the string "/lib/ld-linux.so.2" so I had to make a symbolic link in /lib as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. I have also tried export "LD_POINTER_GUARD=0" also without any change. I have done quite a lot on Modula-3 programming years ago when kernel was 1.2.13. I would like to make public my old sources under the GNU licence. For this reason I am willing to check them on the new version on m3. I am available for doing development in Modula-3 in the limits of my capabilities. My name is Pino Zollo and my e-mail is pinozollo at gmail.com Some of my works are on http://www.qsl.net/zp4kfx and a description of my profesional experience is on http://www.qsl.net/zp4kfx/CV Thanks for the attencion. Best regards Pino ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 18 15:25:04 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 09:25:04 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > It seems I need a check on load of a traced reference in RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once I've checked it in (the CVS mailer is broken again I think). > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:34:58 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:34:58 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: I assume you have an apply method for your RTTypeMap.Visitor that takes "field: ADDRESS" and treats it as "REF REFANY". This is wrong. When reading a REF field you should use the following idiom: WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO ... access field via ref^ ... END; This will automatically insert a call to the appropriate runtime routines on accessing the reference field. There should be no need for you to call the runtime routines directly. On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:40:07 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:40:07 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: PS You should not use "REF REFANY", since a "REF" is assumed by the runtime system to be a *tidy* pointer to an object in the heap. Clearly, the address of a field inside an object is by definition *untidy*. On Apr 18, 2007, at 10:34 AM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:43:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:43:32 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <5E2DE156-C048-4E0B-A471-2A096BABA306@cs.purdue.edu> PS There is no fix to RTTypeMap needed here so long as you use the correct idiom as described earlier. This is unsafe code so you are expected to behave properly by the runtime system. On Apr 18, 2007, at 9:25 AM, Tony Hosking wrote: > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> > > It seems I need a check on load of a traced reference in > RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once > I've checked it in (the CVS mailer is broken again I think). > >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:50:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:50:08 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <5AB076A1-4E10-4BD0-8450-12E7B0F9B7B8@cs.purdue.edu> Does kernel 2.4 support NPTL? On Apr 18, 2007, at 2:04 AM, Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message- > id:from:to:subject:date:mime-version:content-type:content-transfer- > encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/ > WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQM > JJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ > U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 > tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on > plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on > plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following > problem: > > cm3 compiles all programs, but are correctly executed only the ones > compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not > doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did > put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" > so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any > change. > > I have done quite a lot on Modula-3 programming years ago when > kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to > check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a > description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney.bates at wichita.edu Wed Apr 18 18:15:58 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Wed, 18 Apr 2007 11:15:58 -0500 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <4626443E.8040704@wichita.edu> This symptom (works when statically linked, segfaults during startup when using dynamic libraries) is, as I remember, the symptom of the problem with changes in newer setjmp/longjmp that I posted about on January 11. The newer setjmp/longjmp mangle and demangle a couple of the registers (apparently to prevent some security exploit). When used as a pair, the change is harmless. But some uses of setjmp in the M3 libraries actually use the contents of the setjmp buffer to get register contents for thread switching, etc. There is an assembly-coded special version of setjmp in the M3 code, dating from many years back, that works with older longjmp, but not with the newest. Tony committed a change that removes this version of setjmp. but I don't think that fixes all the problems in all cases. I think existing uses of setjmp have to be split into two categories: those that just use the result to pass back to longjmp, and those that use the result for fetching registers. Then different variants of setjmp have to be used in the two cases. Antony suggested getcontext/setcontext, but I guess there are portability problems here? I looked at this briefly, and there are a very large number of uses of setjmp to check out. I have thought about working on it, but have been busy with other things for a while. I never got -DPTHREAD working and ran out of time on that approach. As I recall, -DPTHREAD is now the default, so I would guess Pino's installation is using it. Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following problem: > > cm3 compiles all programs, but are correctly executed only the ones compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any change. > > I have done quite a lot on Modula-3 programming years ago when kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Wed Apr 18 18:33:06 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 12:33:06 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <4626443E.8040704@wichita.edu> References: <20070418060417.GB14308@elegosoft.com> <4626443E.8040704@wichita.edu> Message-ID: Yes, use of setjmp/longjmp for *user*-level threads is bound to fail in the presence of the security mangling they now do, since new thread contexts are obtained by cloning a setjmp buffer from the root thread, and then hacking its stack pointer appropriately to match the new thread. This doesn't work with the secure setjmp/longjmp implementations. On platforms like Linux where setjmp/longjmp mangling prevent their use for user-level threading the only alternative is to use the explicit makecontext with getcontext/ setcontext. This is how *user*-level threading is done on Solaris. With the newer PTHREAD threading (for SOLgnu, SOLsun, LINUXLIBC6, I386_DARWIN, PPC_DARWIN) this should not be an issue, but it does require NPTL on Linux platforms (I don't think 2.4 kernels support NPTL). I do know that I can successfully build from the current CVS tree on recent releases of Fedora Core, without problems. On Apr 18, 2007, at 12:15 PM, Rodney M. Bates wrote: > This symptom (works when statically linked, segfaults during > startup when > using dynamic libraries) is, as I remember, the symptom of the > problem with > changes in newer setjmp/longjmp that I posted about on January 11. > > The newer setjmp/longjmp mangle and demangle a couple of the registers > (apparently to prevent some security exploit). When used as a > pair, the > change is harmless. But some uses of setjmp in the M3 libraries > actually > use the contents of the setjmp buffer to get register contents for > thread > switching, etc. > > There is an assembly-coded special version of setjmp in the M3 > code, dating > from many years back, that works with older longjmp, but not with > the newest. > Tony committed a change that removes this version of setjmp. but I > don't think > that fixes all the problems in all cases. I don't see problems with this change. What problems are you seeing? In this case, setjmp/longjmp is only being used for exception handling, assuming threads are using PTHREAD (i.e., Linux NPTL). > > I think existing uses of setjmp have to be split into two > categories: those > that just use the result to pass back to longjmp, and those that > use the result > for fetching registers. Then different variants of setjmp have to > be used in > the two cases. Antony suggested getcontext/setcontext, but I guess > there are > portability problems here? > > I looked at this briefly, and there are a very large number of uses > of setjmp > to check out. I have thought about working on it, but have been > busy with > other things for a while. > > I never got -DPTHREAD working and ran out of time on that approach. > As I recall, > -DPTHREAD is now the default, so I would guess Pino's installation > is using it. > > Olaf Wagner wrote: >> Is this the stack encryption problem again? Has anybody tried on >> Suse 10.1? >> Olaf >> ----- Forwarded message from Pino Zollo ----- >> DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; >> d=gmail.com; s=beta; >> h=domainkey-signature:received:received:message- >> id:from:to:subject:date:mime-version:content-type:content-transfer- >> encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; >> b=U78B+w8R6V2/ >> WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQ >> MJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ >> U4PrfCQw+NQLqPP2k0Vwncv7Ss= >> From: Pino Zollo >> To: m3-support at elego.de >> Subject: Installation Problem >> Date: Tue, 17 Apr 2007 18:00:50 -0400 >> X-Spam-Status: No, score=2.7 required=5.0 >> tests=DEAR_SOMETHING,RCVD_BY_IP, >> RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no >> version=3.0.3 >> X-Spam-Level: ** >> X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on >> plane.elego.de >> X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on >> plane.elego.de >> X-Virus-Status: Clean >> Dear Sirs, >> I have installed the last version of cm3 but I have the following >> problem: >> cm3 compiles all programs, but are correctly executed only the >> ones compiled >> with the option build_standalone(). >> The programs which use shared libraries end in segmentation fault. >> Also the script do-cm3-std.sh fails to compile everything for the >> same >> problem: the program PklFonts in the directory >> m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault >> because needs shared libraries. >> Any suggestion ? >> I am using the distribution SuSE 10.1 with kernel 2.4.27 >> In the installation I have found that the command "export" was not >> doing >> properly its job, so I had to put >> the path /usr/local/cm3/bin in /etc/profile. For libraries I did >> put the >> path in /etc/ld.so.conf >> I noticed that executables contain the string "/lib/ld-linux.so.2" >> so I had >> to make a symbolic link in /lib >> as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. >> I have also tried export "LD_POINTER_GUARD=0" also without any >> change. >> I have done quite a lot on Modula-3 programming years ago when >> kernel was >> 1.2.13. I would like to make public my >> old sources under the GNU licence. For this reason I am willing to >> check >> them on the new version on m3. >> I am available for doing development in Modula-3 in the limits of my >> capabilities. >> My name is Pino Zollo and my e-mail is pinozollo at gmail.com >> Some of my works are on http://www.qsl.net/zp4kfx and a >> description of my >> profesional experience is on >> http://www.qsl.net/zp4kfx/CV >> Thanks for the attencion. >> Best regards >> Pino >> ----- End forwarded message ----- > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From darko at darko.org Thu Apr 19 03:37:10 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 03:37:10 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Not actually using that code, but I did see your helpful commit on it some time ago. It seems that the problem I described earlier is due to another bug. The code I'm working on does a lot of reading and writing of traced references and I was hoping to get a better understanding of situations I need to be aware of and how to use those RTHooks calls more efficiently. So say I have two traced objects and I am copying a traced reference from a field in one to the other. What set of calls would be normally required there? I'm sure you haven't got time to go into the minutiae of the runtime system, but any hints would be appreciated. Cheers, Darko. On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > From hosking at cs.purdue.edu Thu Apr 19 04:05:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:05:42 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. You SHOULD NOT need to use these calls so long as you are accessing using properly typed references (as in my example earlier). This is not an issue for SAFE code, only in UNSAFE modules must you be careful how you use LOOPHOLE to cast references. > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? How are you copying the references? I would like to see your code for this. So long as you properly type things then the calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be generated by the compiler. Anyway... RTHooks.CheckLoadTracedRef is generated by the compiler whenever you access a traced reference in memory (e.g., via dereference or from a shared variable). Its job is to prevent mutators from acquiring references to "gray" objects (that are reachable but still to be scanned for the traced references that they contain). The check turns the object from gray to black by scanning its references and making sure none of them refer to white objects (that are not known to be reachable by the collector). RTHooks.CheckStoreTraced is generated by the compiler whenever a traced reference is stored (or might be stored, e.g., for VAR) into a traced object. The check makes sure that the generational GC knows that the object has been modified. It needs to know this for objects in the older genration. Again, I reiterate that it is not my intention that programmers actually call these runtime hooks explicitly! I am certain I can rewrite any code you might have so that you do not need to call them explicitly. Instead, properly typed references will result in the checks being generated for you by the compiler. Please feel free to send me code snippets for review. > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From hosking at cs.purdue.edu Thu Apr 19 04:09:35 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:09:35 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <55BF1964-E58E-4E20-93F7-9DDA1A7B1CF4@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. > > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? If you have: VAR source, target: OBJECT field: REFANY END; then copying field from source to target is written as normal: target.field := source.field; The compiler will emit the appropriate calls: CheckLoadTracedRef(source.field) CheckStoreTraced(target) Why do you need to call these runtime routines explicitly if the compiler will generate them for you? > > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From darko at darko.org Thu Apr 19 04:49:29 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 04:49:29 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> Message-ID: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Thanks, that's very helpful. I agree I shouldn't need it but it's exactly because I'm not using the compiler is why I need to know about it. The code is attached, it's still in development and unfinished but you'll get the idea. Basically the module allows you to programatically access structure fields. It allows for applications like being able to create indexes (using an adaptation of the Table code) of objects using arbitrary fields as keys. It makes dynamic programming a bit more dynamic. The code is intended to be completely safe, but there's more checks I need to do. -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.m3 Type: application/octet-stream Size: 41052 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.i3 Type: application/octet-stream Size: 6117 bytes Desc: not available URL: -------------- next part -------------- On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > > On Apr 18, 2007, at 9:37 PM, Darko wrote: > >> Not actually using that code, but I did see your helpful commit on >> it some time ago. It seems that the problem I described earlier is >> due to another bug. The code I'm working on does a lot of reading >> and writing of traced references and I was hoping to get a better >> understanding of situations I need to be aware of and how to use >> those RTHooks calls more efficiently. > > You SHOULD NOT need to use these calls so long as you are accessing > using properly typed references (as in my example earlier). This > is not an issue for SAFE code, only in UNSAFE modules must you be > careful how you use LOOPHOLE to cast references. > >> So say I have two traced objects and I am copying a traced >> reference from a field in one to the other. What set of calls >> would be normally required there? > > How are you copying the references? I would like to see your code > for this. So long as you properly type things then the calls to > RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be > generated by the compiler. > > Anyway... > > RTHooks.CheckLoadTracedRef is generated by the compiler whenever > you access a traced reference in memory (e.g., via dereference or > from a shared variable). Its job is to prevent mutators from > acquiring references to "gray" objects (that are reachable but > still to be scanned for the traced references that they contain). > The check turns the object from gray to black by scanning its > references and making sure none of them refer to white objects > (that are not known to be reachable by the collector). > > RTHooks.CheckStoreTraced is generated by the compiler whenever a > traced reference is stored (or might be stored, e.g., for VAR) into > a traced object. The check makes sure that the generational GC > knows that the object has been modified. It needs to know this for > objects in the older genration. > > Again, I reiterate that it is not my intention that programmers > actually call these runtime hooks explicitly! I am certain I can > rewrite any code you might have so that you do not need to call > them explicitly. Instead, properly typed references will result in > the checks being generated for you by the compiler. Please feel > free to send me code snippets for review. > >> I'm sure you haven't got time to go into the minutiae of the >> runtime system, but any hints would be appreciated. >> >> Cheers, >> Darko. >> >> >> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >> >>> I assume you have an apply method for your RTTypeMap.Visitor that >>> takes "field: ADDRESS" and treats it as "REF REFANY". This is >>> wrong. When reading a REF field you should use the following idiom: >>> >>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>> ... access field via ref^ ... >>> END; >>> >>> This will automatically insert a call to the appropriate runtime >>> routines on accessing the reference field. >>> >>> There should be no need for you to call the runtime routines >>> directly. >>> >>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>> >>>> Hi, >>>> >>>> Wondering if you can explain the use of these calls a little >>>> more. I'm currently using type maps to read and write fields >>>> from traced objects. Reading a traced reference from inside a >>>> traced object into a local variable is not working as it should. >>>> Should I use CheckLoadTraced and if so when and how? Looking at >>>> your changes to RTTypeMap, writing references into objects means >>>> you need to call CheckStoreTraced on the object written inside >>>> of, before it is written? >>>> >>>> Cheers, >>>> Darko. >>> > From hosking at cs.purdue.edu Thu Apr 19 15:34:40 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 09:34:40 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Message-ID: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> My immediate reaction is that you probably want to approach reads slightly differently. Instead of using your generic InTypeRead routine to access the appropriate bits/bytes of the value, you probably want to replace it with some sort of simple addressing mechanism that computes the raw address of the field. Traced reference fields must always be aligned within the heap object they are stored in so your addressing for these will not need bit offsets. Once you have an ADDRESS "addr" for the traced reference field, so long as you use the idiom I gave earlier, you will get the correct behavior for reads. That is, WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO .. use field^ to access the traced reference stored at address field .. END; The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the traced reference "r" once you have loopholed it through the raw field access into a REFANY, before you return it from InTypeReadRef: PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): REFANY = VAR v: REFANY; BEGIN this.check(field, Kind.Ref, obj); this.read(obj, field, ADR(v), BYTESIZE(v)); RTHooks.CheckLoadTracedRef(v); RETURN v; END InTypeReadRef; This is safe for the current CM3 ambiguous roots collector, but this code would probably break with a fully copying/relocating collector (if one was ever implemented for Modula-3), since the raw bytes of the reference might become stale between the point you load them from memory and the point they are placed in the typed REFANY variable "v" (effectively, you'd be hiding a reference from the GC for a brief period, and it would not be able to update that hidden reference in the face of its target being moved). I note, generally, that your field addressing mechanisms only work anyway because we are using an ambiguous roots collector that pins all objects referenced from the thread stacks (your "obj" parameters in the accessor methods), which allows the use of RTHeap.GetDataAdr to provide an address for the object that is valid while the object reference is held on the stack. When writing references to object fields you will need to invoke "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is being stored. Thus, for InTypeWriteRef: PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: CARDINAL) = BEGIN this.check(field, Kind.Ref, obj); WITH f = this.offs.get(field) DO IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; END; this.write(obj, field, ADR(val), BYTESIZE(val)); RTHooks.CheckStoreTraced(obj); END InTypeWriteRef; If you are performing multiple writes of reference fields on the same object (e.g., to an array) then you can invoke CheckStoreTraced once for the whole object. Hope this helps. On Apr 18, 2007, at 10:49 PM, Darko wrote: > Thanks, that's very helpful. I agree I shouldn't need it but it's > exactly because I'm not using the compiler is why I need to know > about it. The code is attached, it's still in development and > unfinished but you'll get the idea. > > Basically the module allows you to programatically access structure > fields. It allows for applications like being able to create > indexes (using an adaptation of the Table code) of objects using > arbitrary fields as keys. It makes dynamic programming a bit more > dynamic. The code is intended to be completely safe, but there's > more checks I need to do. > > > > > > > > > > On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > >> >> On Apr 18, 2007, at 9:37 PM, Darko wrote: >> >>> Not actually using that code, but I did see your helpful commit >>> on it some time ago. It seems that the problem I described >>> earlier is due to another bug. The code I'm working on does a lot >>> of reading and writing of traced references and I was hoping to >>> get a better understanding of situations I need to be aware of >>> and how to use those RTHooks calls more efficiently. >> >> You SHOULD NOT need to use these calls so long as you are >> accessing using properly typed references (as in my example >> earlier). This is not an issue for SAFE code, only in UNSAFE >> modules must you be careful how you use LOOPHOLE to cast references. >> >>> So say I have two traced objects and I am copying a traced >>> reference from a field in one to the other. What set of calls >>> would be normally required there? >> >> How are you copying the references? I would like to see your code >> for this. So long as you properly type things then the calls to >> RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be >> generated by the compiler. >> >> Anyway... >> >> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >> you access a traced reference in memory (e.g., via dereference or >> from a shared variable). Its job is to prevent mutators from >> acquiring references to "gray" objects (that are reachable but >> still to be scanned for the traced references that they contain). >> The check turns the object from gray to black by scanning its >> references and making sure none of them refer to white objects >> (that are not known to be reachable by the collector). >> >> RTHooks.CheckStoreTraced is generated by the compiler whenever a >> traced reference is stored (or might be stored, e.g., for VAR) >> into a traced object. The check makes sure that the generational >> GC knows that the object has been modified. It needs to know this >> for objects in the older genration. >> >> Again, I reiterate that it is not my intention that programmers >> actually call these runtime hooks explicitly! I am certain I can >> rewrite any code you might have so that you do not need to call >> them explicitly. Instead, properly typed references will result >> in the checks being generated for you by the compiler. Please >> feel free to send me code snippets for review. >> >>> I'm sure you haven't got time to go into the minutiae of the >>> runtime system, but any hints would be appreciated. >>> >>> Cheers, >>> Darko. >>> >>> >>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>> >>>> I assume you have an apply method for your RTTypeMap.Visitor >>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>> This is wrong. When reading a REF field you should use the >>>> following idiom: >>>> >>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>> ... access field via ref^ ... >>>> END; >>>> >>>> This will automatically insert a call to the appropriate runtime >>>> routines on accessing the reference field. >>>> >>>> There should be no need for you to call the runtime routines >>>> directly. >>>> >>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>> >>>>> Hi, >>>>> >>>>> Wondering if you can explain the use of these calls a little >>>>> more. I'm currently using type maps to read and write fields >>>>> from traced objects. Reading a traced reference from inside a >>>>> traced object into a local variable is not working as it >>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>> Looking at your changes to RTTypeMap, writing references into >>>>> objects means you need to call CheckStoreTraced on the object >>>>> written inside of, before it is written? >>>>> >>>>> Cheers, >>>>> Darko. >>>> >> > From darko at darko.org Thu Apr 19 19:09:45 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 19:09:45 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: Thanks, thats very useful. Garbage collection is a bit of a mystery, since I can't imagine myself changing it I've never studied it, but it's on my mind constantly. M3 is so safe whenever something weird happens with my code I know it's because I've trodden on a traced reference, or copied it into a untraced space and back or something. I've relied on the pinning of references that appear in the stack extensively, and I think a lot of other code does too. It a bit of a worry to think that it's possible it might disappear. If we don't have this, short of stopping the collector, what do you do if you want your object to stay still? On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > My immediate reaction is that you probably want to approach reads > slightly differently. Instead of using your generic InTypeRead > routine to access the appropriate bits/bytes of the value, you > probably want to replace it with some sort of simple addressing > mechanism that computes the raw address of the field. Traced > reference fields must always be aligned within the heap object they > are stored in so your addressing for these will not need bit > offsets. Once you have an ADDRESS "addr" for the traced reference > field, so long as you use the idiom I gave earlier, you will get > the correct behavior for reads. That is, > > WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO > .. use field^ to access the traced reference stored at address > field .. > END; > > The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the > traced reference "r" once you have loopholed it through the raw > field access into a REFANY, before you return it from InTypeReadRef: > > PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): > REFANY = > VAR v: REFANY; > BEGIN > this.check(field, Kind.Ref, obj); > this.read(obj, field, ADR(v), BYTESIZE(v)); > RTHooks.CheckLoadTracedRef(v); > RETURN v; > END InTypeReadRef; > > This is safe for the current CM3 ambiguous roots collector, but > this code would probably break with a fully copying/relocating > collector (if one was ever implemented for Modula-3), since the raw > bytes of the reference might become stale between the point you > load them from memory and the point they are placed in the typed > REFANY variable "v" (effectively, you'd be hiding a reference from > the GC for a brief period, and it would not be able to update that > hidden reference in the face of its target being moved). > > I note, generally, that your field addressing mechanisms only work > anyway because we are using an ambiguous roots collector that pins > all objects referenced from the thread stacks (your "obj" > parameters in the accessor methods), which allows the use of > RTHeap.GetDataAdr to provide an address for the object that is > valid while the object reference is held on the stack. > > When writing references to object fields you will need to invoke > "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is > being stored. Thus, for InTypeWriteRef: > > PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: > CARDINAL) = > BEGIN > this.check(field, Kind.Ref, obj); > WITH f = this.offs.get(field) DO > IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE > (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; > END; > this.write(obj, field, ADR(val), BYTESIZE(val)); > RTHooks.CheckStoreTraced(obj); > END InTypeWriteRef; > > If you are performing multiple writes of reference fields on the > same object (e.g., to an array) then you can invoke > CheckStoreTraced once for the whole object. > > Hope this helps. > > On Apr 18, 2007, at 10:49 PM, Darko wrote: > >> Thanks, that's very helpful. I agree I shouldn't need it but it's >> exactly because I'm not using the compiler is why I need to know >> about it. The code is attached, it's still in development and >> unfinished but you'll get the idea. >> >> Basically the module allows you to programatically access >> structure fields. It allows for applications like being able to >> create indexes (using an adaptation of the Table code) of objects >> using arbitrary fields as keys. It makes dynamic programming a bit >> more dynamic. The code is intended to be completely safe, but >> there's more checks I need to do. >> >> >> >> >> >> >> >> >> >> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >> >>> >>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>> >>>> Not actually using that code, but I did see your helpful commit >>>> on it some time ago. It seems that the problem I described >>>> earlier is due to another bug. The code I'm working on does a >>>> lot of reading and writing of traced references and I was hoping >>>> to get a better understanding of situations I need to be aware >>>> of and how to use those RTHooks calls more efficiently. >>> >>> You SHOULD NOT need to use these calls so long as you are >>> accessing using properly typed references (as in my example >>> earlier). This is not an issue for SAFE code, only in UNSAFE >>> modules must you be careful how you use LOOPHOLE to cast references. >>> >>>> So say I have two traced objects and I am copying a traced >>>> reference from a field in one to the other. What set of calls >>>> would be normally required there? >>> >>> How are you copying the references? I would like to see your >>> code for this. So long as you properly type things then the >>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>> will be generated by the compiler. >>> >>> Anyway... >>> >>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>> you access a traced reference in memory (e.g., via dereference or >>> from a shared variable). Its job is to prevent mutators from >>> acquiring references to "gray" objects (that are reachable but >>> still to be scanned for the traced references that they >>> contain). The check turns the object from gray to black by >>> scanning its references and making sure none of them refer to >>> white objects (that are not known to be reachable by the collector). >>> >>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>> traced reference is stored (or might be stored, e.g., for VAR) >>> into a traced object. The check makes sure that the generational >>> GC knows that the object has been modified. It needs to know >>> this for objects in the older genration. >>> >>> Again, I reiterate that it is not my intention that programmers >>> actually call these runtime hooks explicitly! I am certain I can >>> rewrite any code you might have so that you do not need to call >>> them explicitly. Instead, properly typed references will result >>> in the checks being generated for you by the compiler. Please >>> feel free to send me code snippets for review. >>> >>>> I'm sure you haven't got time to go into the minutiae of the >>>> runtime system, but any hints would be appreciated. >>>> >>>> Cheers, >>>> Darko. >>>> >>>> >>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>> >>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>> This is wrong. When reading a REF field you should use the >>>>> following idiom: >>>>> >>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>> ... access field via ref^ ... >>>>> END; >>>>> >>>>> This will automatically insert a call to the appropriate >>>>> runtime routines on accessing the reference field. >>>>> >>>>> There should be no need for you to call the runtime routines >>>>> directly. >>>>> >>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Wondering if you can explain the use of these calls a little >>>>>> more. I'm currently using type maps to read and write fields >>>>>> from traced objects. Reading a traced reference from inside a >>>>>> traced object into a local variable is not working as it >>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>> objects means you need to call CheckStoreTraced on the object >>>>>> written inside of, before it is written? >>>>>> >>>>>> Cheers, >>>>>> Darko. >>>>> >>> >> > From dragisha at m3w.org Thu Apr 19 19:24:57 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 19:24:57 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's Message-ID: <1177003497.11459.17.camel@faramir.m3w.org> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box (Pentium M). It breaks somewhere where PklFonts is to be called. I am not sure if it's because of built binary being started w/ my glibc, or some other issue. Is cm3-5.4.0 possible to build on box like this? Thanks, dd -- Dragi?a Duri? From hosking at cs.purdue.edu Thu Apr 19 19:39:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:39:55 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: On Apr 19, 2007, at 1:09 PM, Darko wrote: > Thanks, thats very useful. Garbage collection is a bit of a > mystery, since I can't imagine myself changing it I've never > studied it, but it's on my mind constantly. M3 is so safe whenever > something weird happens with my code I know it's because I've > trodden on a traced reference, or copied it into a untraced space > and back or something. Yeah, it can be a problem, but if you are careful it should not be too hard to avoid. The idea is not to "hide" a reference where the GC cannot find it. > > I've relied on the pinning of references that appear in the stack > extensively, and I think a lot of other code does too. It a bit of > a worry to think that it's possible it might disappear. If we don't > have this, short of stopping the collector, what do you do if you > want your object to stay still? In the absence of implicit pinning of references from the stack we would need to introduce an explicit pin/unpin API for programmers to use. It is unlikely that CM3 will ever move away from implicit pinning unless we were to build a "smart" backend compiler (ie, not based on gcc). So, short answer is that you are probably OK for a long time to come. > > > On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > >> My immediate reaction is that you probably want to approach reads >> slightly differently. Instead of using your generic InTypeRead >> routine to access the appropriate bits/bytes of the value, you >> probably want to replace it with some sort of simple addressing >> mechanism that computes the raw address of the field. Traced >> reference fields must always be aligned within the heap object >> they are stored in so your addressing for these will not need bit >> offsets. Once you have an ADDRESS "addr" for the traced reference >> field, so long as you use the idiom I gave earlier, you will get >> the correct behavior for reads. That is, >> >> WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO >> .. use field^ to access the traced reference stored at address >> field .. >> END; >> >> The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on >> the traced reference "r" once you have loopholed it through the >> raw field access into a REFANY, before you return it from >> InTypeReadRef: >> >> PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): >> REFANY = >> VAR v: REFANY; >> BEGIN >> this.check(field, Kind.Ref, obj); >> this.read(obj, field, ADR(v), BYTESIZE(v)); >> RTHooks.CheckLoadTracedRef(v); >> RETURN v; >> END InTypeReadRef; >> >> This is safe for the current CM3 ambiguous roots collector, but >> this code would probably break with a fully copying/relocating >> collector (if one was ever implemented for Modula-3), since the >> raw bytes of the reference might become stale between the point >> you load them from memory and the point they are placed in the >> typed REFANY variable "v" (effectively, you'd be hiding a >> reference from the GC for a brief period, and it would not be able >> to update that hidden reference in the face of its target being >> moved). >> >> I note, generally, that your field addressing mechanisms only work >> anyway because we are using an ambiguous roots collector that pins >> all objects referenced from the thread stacks (your "obj" >> parameters in the accessor methods), which allows the use of >> RTHeap.GetDataAdr to provide an address for the object that is >> valid while the object reference is held on the stack. >> >> When writing references to object fields you will need to invoke >> "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field >> is being stored. Thus, for InTypeWriteRef: >> >> PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: >> CARDINAL) = >> BEGIN >> this.check(field, Kind.Ref, obj); >> WITH f = this.offs.get(field) DO >> IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE >> (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; >> END; >> this.write(obj, field, ADR(val), BYTESIZE(val)); >> RTHooks.CheckStoreTraced(obj); >> END InTypeWriteRef; >> >> If you are performing multiple writes of reference fields on the >> same object (e.g., to an array) then you can invoke >> CheckStoreTraced once for the whole object. >> >> Hope this helps. >> >> On Apr 18, 2007, at 10:49 PM, Darko wrote: >> >>> Thanks, that's very helpful. I agree I shouldn't need it but it's >>> exactly because I'm not using the compiler is why I need to know >>> about it. The code is attached, it's still in development and >>> unfinished but you'll get the idea. >>> >>> Basically the module allows you to programatically access >>> structure fields. It allows for applications like being able to >>> create indexes (using an adaptation of the Table code) of objects >>> using arbitrary fields as keys. It makes dynamic programming a >>> bit more dynamic. The code is intended to be completely safe, but >>> there's more checks I need to do. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >>> >>>> >>>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>>> >>>>> Not actually using that code, but I did see your helpful commit >>>>> on it some time ago. It seems that the problem I described >>>>> earlier is due to another bug. The code I'm working on does a >>>>> lot of reading and writing of traced references and I was >>>>> hoping to get a better understanding of situations I need to be >>>>> aware of and how to use those RTHooks calls more efficiently. >>>> >>>> You SHOULD NOT need to use these calls so long as you are >>>> accessing using properly typed references (as in my example >>>> earlier). This is not an issue for SAFE code, only in UNSAFE >>>> modules must you be careful how you use LOOPHOLE to cast >>>> references. >>>> >>>>> So say I have two traced objects and I am copying a traced >>>>> reference from a field in one to the other. What set of calls >>>>> would be normally required there? >>>> >>>> How are you copying the references? I would like to see your >>>> code for this. So long as you properly type things then the >>>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>>> will be generated by the compiler. >>>> >>>> Anyway... >>>> >>>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>>> you access a traced reference in memory (e.g., via dereference >>>> or from a shared variable). Its job is to prevent mutators from >>>> acquiring references to "gray" objects (that are reachable but >>>> still to be scanned for the traced references that they >>>> contain). The check turns the object from gray to black by >>>> scanning its references and making sure none of them refer to >>>> white objects (that are not known to be reachable by the >>>> collector). >>>> >>>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>>> traced reference is stored (or might be stored, e.g., for VAR) >>>> into a traced object. The check makes sure that the >>>> generational GC knows that the object has been modified. It >>>> needs to know this for objects in the older genration. >>>> >>>> Again, I reiterate that it is not my intention that programmers >>>> actually call these runtime hooks explicitly! I am certain I >>>> can rewrite any code you might have so that you do not need to >>>> call them explicitly. Instead, properly typed references will >>>> result in the checks being generated for you by the compiler. >>>> Please feel free to send me code snippets for review. >>>> >>>>> I'm sure you haven't got time to go into the minutiae of the >>>>> runtime system, but any hints would be appreciated. >>>>> >>>>> Cheers, >>>>> Darko. >>>>> >>>>> >>>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>>> >>>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>>> This is wrong. When reading a REF field you should use the >>>>>> following idiom: >>>>>> >>>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>>> ... access field via ref^ ... >>>>>> END; >>>>>> >>>>>> This will automatically insert a call to the appropriate >>>>>> runtime routines on accessing the reference field. >>>>>> >>>>>> There should be no need for you to call the runtime routines >>>>>> directly. >>>>>> >>>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Wondering if you can explain the use of these calls a little >>>>>>> more. I'm currently using type maps to read and write fields >>>>>>> from traced objects. Reading a traced reference from inside a >>>>>>> traced object into a local variable is not working as it >>>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>>> objects means you need to call CheckStoreTraced on the object >>>>>>> written inside of, before it is written? >>>>>>> >>>>>>> Cheers, >>>>>>> Darko. >>>>>> >>>> >>> >> From hosking at cs.purdue.edu Thu Apr 19 19:41:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:41:08 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177003497.11459.17.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: I just bootstrapped cm3 from the CVS head last night, on the most recent up-to-date Fedora core, so I don't know what your problems could be. I wonder if you have a broken bootstrap compiler? On Apr 19, 2007, at 1:24 PM, Dragi?a Duri? wrote: > I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > (Pentium > M). It breaks somewhere where PklFonts is to be called. I am not > sure if > it's because of built binary being started w/ my glibc, or some other > issue. > > Is cm3-5.4.0 possible to build on box like this? > > Thanks, > dd > > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Thu Apr 19 19:55:10 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Thu, 19 Apr 2007 19:55:10 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: <20070419175510.GA16758@elegosoft.com> On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > I just bootstrapped cm3 from the CVS head last night, on the most > recent up-to-date Fedora core, so I don't know what your problems > could be. I wonder if you have a broken bootstrap compiler? > > On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > > >I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >(Pentium > >M). It breaks somewhere where PklFonts is to be called. I am not > >sure if > >it's because of built binary being started w/ my glibc, or some other > >issue. > > > >Is cm3-5.4.0 possible to build on box like this? Hallo Antony, could you provide a minimal binary installation package for LINUXLIBC6 with PTHREADs that we could put on our server for download? I assume many people will run into problems on Linux with the current release. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Thu Apr 19 22:27:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 16:27:42 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419175510.GA16758@elegosoft.com> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: >> I just bootstrapped cm3 from the CVS head last night, on the most >> recent up-to-date Fedora core, so I don't know what your problems >> could be. I wonder if you have a broken bootstrap compiler? >> >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: >> >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box >>> (Pentium >>> M). It breaks somewhere where PklFonts is to be called. I am not >>> sure if >>> it's because of built binary being started w/ my glibc, or some >>> other >>> issue. >>> >>> Is cm3-5.4.0 possible to build on box like this? > > Hallo Antony, > > could you provide a minimal binary installation package for LINUXLIBC6 > with PTHREADs that we could put on our server for download? I assume > many people will run into problems on Linux with the current release. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Thu Apr 19 23:01:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 23:01:20 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: <1177016480.4282.13.camel@faramir.m3w.org> I will try this one... But I am not sure we understand each other here... Bootstrap compiler from download site works for me, except for problems with PklFonts and mentor, also met by other ppl... I will look in that sometimes, as Trestle apps are not of primary interest for me anyway... Problem happens when trying to run some executable, I'll paste fisheye example... faramir:dragisha/pts/7: ~/cm3-inst% fisheye zsh: segmentation fault fisheye It is what happens. I must fix it with: faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye Is this only way to run cm3 built executables under modern Linuces? dd On Thu, 2007-04-19 at 16:27 -0400, Tony Hosking wrote: > There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > > > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > >> I just bootstrapped cm3 from the CVS head last night, on the most > >> recent up-to-date Fedora core, so I don't know what your problems > >> could be. I wonder if you have a broken bootstrap compiler? > >> > >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > >> > >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >>> (Pentium > >>> M). It breaks somewhere where PklFonts is to be called. I am not > >>> sure if > >>> it's because of built binary being started w/ my glibc, or some > >>> other > >>> issue. > >>> > >>> Is cm3-5.4.0 possible to build on box like this? > > > > Hallo Antony, > > > > could you provide a minimal binary installation package for LINUXLIBC6 > > with PTHREADs that we could put on our server for download? I assume > > many people will run into problems on Linux with the current release. > > > > Olaf > > -- > > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > > Berlin, Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > > 23 45 86 95 > > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > > Sitz: Berlin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > > DE163214194 > -- Dragi?a Duri? From stsp at elego.de Thu Apr 19 23:05:09 2007 From: stsp at elego.de (Stefan Sperling) Date: Thu, 19 Apr 2007 23:05:09 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177016480.4282.13.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> Message-ID: <20070419210509.GB11507@jack.stsp.lan> On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: > Problem happens when trying to run some executable, I'll paste fisheye > example... > > faramir:dragisha/pts/7: ~/cm3-inst% fisheye > zsh: segmentation fault fisheye > > It is what happens. I must fix it with: > > faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye > > Is this only way to run cm3 built executables under modern Linuces? Yes. It's because of a change in recent glibc versions. I forgot the specifics, I guess someone else will have to elaborate :-P -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From hosking at cs.purdue.edu Thu Apr 19 23:44:20 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 17:44:20 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419210509.GB11507@jack.stsp.lan> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: Download and unzip the bootstrap compiler executable from ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. Rebuild using this bootstrap compiler from the most recent CVS sources checked out from elegosoft. This will build a pthreads-enabled run-time system that avoids the pointer guard problems with user-threading based on setjmp/longjmp. I verified this build for me just last night on the most recent release of Fedora Core. On Apr 19, 2007, at 5:05 PM, Stefan Sperling wrote: > On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: >> Problem happens when trying to run some executable, I'll paste >> fisheye >> example... >> >> faramir:dragisha/pts/7: ~/cm3-inst% fisheye >> zsh: segmentation fault fisheye >> >> It is what happens. I must fix it with: >> >> faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye >> >> Is this only way to run cm3 built executables under modern Linuces? > > Yes. It's because of a change in recent glibc versions. > I forgot the specifics, I guess someone else will have to > elaborate :-P > -- > stefan > http://stsp.name PGP Key: > 0xF59D25F0 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Fri Apr 20 07:59:02 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Fri, 20 Apr 2007 07:59:02 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: <20070420055902.GA1418@elegosoft.com> On Thu, Apr 19, 2007 at 05:44:20PM -0400, Tony Hosking wrote: > Download and unzip the bootstrap compiler executable from ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > Rebuild using this bootstrap compiler from the most recent CVS > sources checked out from elegosoft. > > This will build a pthreads-enabled run-time system that avoids the > pointer guard problems with user-threading based on setjmp/longjmp. > > I verified this build for me just last night on the most recent > release of Fedora Core. Hi Stefan, I just checked our known problem pages, and the problem with the segment violation in indeed already documented. Could you additionally put up Antony's short description together with his link for a current CM3 bootstrap? Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 09:25:38 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 09:25:38 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. Message-ID: <1177053938.4282.29.camel@faramir.m3w.org> While maintaining our spinoff pm3 version, I've cleaned most of syscall wrapper needs from library source. Problem during early m3core development was, most probably, unclean situation about where and how C library will be invoked and someone decided to play safe, and to wrap all syscalls implied by used library calls to be fully safe. Across m3core and libm3, number of these low level, C library calls is really small. File I/O, socket I/O, filesystem operations... make a big majority of them (more than 98%). As all of these low level C lib calls these were easily identifiable (few grep's), I've cleaned them without much problems and after that, I've eliminated wrapper by wrapper. At this moment, I think none is present in our in-house m3 env. As time permits, and unless none else makes these steps, I will repeat them for CM3. I know answer to this following question is hidden somewhere in archives, but as my previous questions has shown - it can be hidden very deep :). So, can someone highlight current m3gc and syscall wrappers situation for me? How do I build CM3 WITH most extensive GC available at the moment? dd P.S. Big thanks to Tony Hosking for LD_POINTER_GUARD! As I have many Modula-3 apps in production use, this was very big issue for me. -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 10:07:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 10:07:20 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles Message-ID: <1177056440.4282.34.camel@faramir.m3w.org> while trying to make my programs with newsly installed cm3, I am constantly running into same error. Example: import("libm3") include_dir("../../../../m3lib0/src") include_dir("../../../../pdf/src") include_dir("../../../../zip/src") implementation("Test") program("test") Is my m3makefile, and when "cm3" is run in package's src dir, I have only: faramir:dragisha/pts/9: test/1/src% cm3 --- building in ../LINUXLIBC6 --- *** *** runtime error: *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES list *** file "../src/os/POSIX/PathnamePosix.m3", line 98 *** Mentioned line raises this FATAL exception when Absolute(path) is TRUE... What is absolute in my paths? All m3makefiles in question were ok with pm3, of course.. dd -- Dragi?a Duri? From wagner at elegosoft.com Fri Apr 20 11:57:34 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 11:57:34 +0200 (CEST) Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <1177053938.4282.29.camel@faramir.m3w.org> References: <1177053938.4282.29.camel@faramir.m3w.org> Message-ID: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: > While maintaining our spinoff pm3 version, I've cleaned most of syscall > wrapper needs from library source. Problem during early m3core > development was, most probably, unclean situation about where and how C > library will be invoked and someone decided to play safe, and to wrap > all syscalls implied by used library calls to be fully safe. > > Across m3core and libm3, number of these low level, C library calls is > really small. File I/O, socket I/O, filesystem operations... make a big > majority of them (more than 98%). As all of these low level C lib calls > these were easily identifiable (few grep's), I've cleaned them without > much problems and after that, I've eliminated wrapper by wrapper. At > this moment, I think none is present in our in-house m3 env. You would also need to consider all indirect system calls via libraries and embedded C code. I think the M3 developers tried to offer a more or less complete set of safe system calls, regardless how they are called from the M3 program. > As time > permits, and unless none else makes these steps, I will repeat them for > CM3. I don't think this is a good idea right now. > I know answer to this following question is hidden somewhere in > archives, but as my previous questions has shown - it can be hidden very > deep :). So, can someone highlight current m3gc and syscall wrappers > situation for me? How do I build CM3 WITH most extensive GC available at > the moment? The current CM3 compiler/runtime does not need virtual memory protection and thus does not need the system call wrappers any more, as Antony has modified the compiler to generate hints for the garbage collector. So you can more or less ignore this issue (unless you really want memory protection) and simply use m3gc-simple for all platforms. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 14:21:19 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:19 +0200 Subject: [M3devel] cm3 pthread problem? Message-ID: <1177071680.4282.47.camel@faramir.m3w.org> Attached program won't run under freshly built cm3 @ Fedora Core 6. Compiles, runs... and stucks... It works flawlessly with user-level threads and also with mine implementation of NPTL for pm3. I've done it for 4000 threads per 10000 passess, and this version I am attaching, and trying without success on CM3 is 40 threads... Of course, before I run it, I am ulimiting stack size to 64kB or similar.. It fits. dd -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 14:21:56 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:56 +0200 Subject: [M3devel] missing attachment, sorry Message-ID: <1177071717.4282.49.camel@faramir.m3w.org> -- Dragi?a Duri? -------------- next part -------------- MODULE TestThreads EXPORTS Main; IMPORT Fmt, IO, Thread; TYPE MyClosure = Thread.Closure OBJECT my, next: Thread.Condition; OVERRIDES apply := JustDoIt; END; VAR glob := NEW(MUTEX); started: CARDINAL; PROCEDURE JustDoIt(c: MyClosure): REFANY = BEGIN LOCK glob DO INC(started); Thread.Wait(glob, c.my); Thread.Signal(c.next); END; RETURN NIL; END JustDoIt; CONST nThreads = 40; VAR my, first, a, b: Thread.Condition; BEGIN FOR j:=1 TO 10 DO IO.Put("Forking " & Fmt.Int(nThreads) & " threads... (" & Fmt.Int(j) & ")\n"); IF j MOD 10 = 0 THEN IO.Put("(" & Fmt.Int(j) & ")"); END; my := NEW(Thread.Condition); a:=NEW(Thread.Condition); started:=0; FOR i:=1 TO nThreads DO IF i=nThreads THEN b:=my; ELSE b:=NEW(Thread.Condition); END; EVAL Thread.Fork(NEW(MyClosure, my := a, next := b)); IF i=1 THEN first:=a; END; a:=b; END; IO.Put("Forked.\n"); IO.Put(","); WHILE started References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> Message-ID: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 1:10 pm, Dragi??a Duri?? wrote: > On Fri, 2007-04-20 at 11:57 +0200, Olaf Wagner wrote: >> On Fri, April 20, 2007 9:25 am, Dragi? ??a Duri????? wrote: >> > While maintaining our spinoff pm3 version, I've cleaned most of syscall >> > wrapper needs from library source. Problem during early m3core >> > development was, most probably, unclean situation about where and how C >> > library will be invoked and someone decided to play safe, and to wrap >> > all syscalls implied by used library calls to be fully safe. >> > >> > Across m3core and libm3, number of these low level, C library calls is >> > really small. File I/O, socket I/O, filesystem operations... make a big >> > majority of them (more than 98%). As all of these low level C lib calls >> > these were easily identifiable (few grep's), I've cleaned them without >> > much problems and after that, I've eliminated wrapper by wrapper. At >> > this moment, I think none is present in our in-house m3 env. >> >> You would also need to consider all indirect system calls via libraries >> and embedded C code. I think the M3 developers tried to offer a more or >> less complete set of safe system calls, regardless how they are called >> from the M3 program. > > It is, but C libraries are mostly in these few percents remaining... > Good wrapping techniques and auditing of existing libraries (hint: grep > EXTERNAL) would be small effort for seamless integration of full m3 > runtime with unperfect C world. I was trying to say that cover only the calls from libraries now used by the M3 runtime, but M3 is an open language and allows the integration of C and C++ (and other) code (and thus of arbitrary other libraries). >> > I know answer to this following question is hidden somewhere in >> > archives, but as my previous questions has shown - it can be hidden very >> > deep :). So, can someone highlight current m3gc and syscall wrappers >> > situation for me? How do I build CM3 WITH most extensive GC available at >> > the moment? >> >> The current CM3 compiler/runtime does not need virtual memory protection >> and thus does not need the system call wrappers any more, as Antony has >> modified the compiler to generate hints for the garbage collector. So >> you can more or less ignore this issue (unless you really want memory >> protection) and simply use m3gc-simple for all platforms. > > Does it mean garbage collection is not > incremental/generational/supercallifraglistic... anymore, or Anthony's > work has made it work without memory protection? The latter; we've got incremental and generational garbage collection without the need for memory protection and system call wrappers :) Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Fri Apr 20 14:59:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 08:59:32 -0400 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> Message-ID: <9B2640F3-10E7-4CE6-BB63-32090198FA1C@cs.purdue.edu> Yes, I concur that there is no need to expend any effort on syscall wrappers. I plan to eliminate the legacy syscall wrappers from all CM3 targets very soon, since the new GC no longer needs them. I would hate to see the library sources cluttered with unnecessary changes that relate only to the deprecated syscall wrappers. On Apr 20, 2007, at 5:57 AM, Olaf Wagner wrote: > > On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: >> While maintaining our spinoff pm3 version, I've cleaned most of >> syscall >> wrapper needs from library source. Problem during early m3core >> development was, most probably, unclean situation about where and >> how C >> library will be invoked and someone decided to play safe, and to wrap >> all syscalls implied by used library calls to be fully safe. >> >> Across m3core and libm3, number of these low level, C library >> calls is >> really small. File I/O, socket I/O, filesystem operations... make >> a big >> majority of them (more than 98%). As all of these low level C lib >> calls >> these were easily identifiable (few grep's), I've cleaned them >> without >> much problems and after that, I've eliminated wrapper by wrapper. At >> this moment, I think none is present in our in-house m3 env. > > You would also need to consider all indirect system calls via > libraries > and embedded C code. I think the M3 developers tried to offer a > more or > less complete set of safe system calls, regardless how they are called > from the M3 program. > >> As time >> permits, and unless none else makes these steps, I will repeat >> them for >> CM3. > > I don't think this is a good idea right now. > >> I know answer to this following question is hidden somewhere in >> archives, but as my previous questions has shown - it can be >> hidden very >> deep :). So, can someone highlight current m3gc and syscall wrappers >> situation for me? How do I build CM3 WITH most extensive GC >> available at >> the moment? > > The current CM3 compiler/runtime does not need virtual memory > protection > and thus does not need the system call wrappers any more, as Antony > has > modified the compiler to generate hints for the garbage collector. So > you can more or less ignore this issue (unless you really want memory > protection) and simply use m3gc-simple for all platforms. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Fri Apr 20 15:06:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 15:06:11 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> Message-ID: <1177074372.4282.54.camel@faramir.m3w.org> On Fri, 2007-04-20 at 14:56 +0200, Olaf Wagner wrote: ... > >> You would also need to consider all indirect system calls via libraries > >> and embedded C code. I think the M3 developers tried to offer a more or > >> less complete set of safe system calls, regardless how they are called > >> from the M3 program. > > > > It is, but C libraries are mostly in these few percents remaining... > > Good wrapping techniques and auditing of existing libraries (hint: grep > > EXTERNAL) would be small effort for seamless integration of full m3 > > runtime with unperfect C world. > > I was trying to say that cover only the calls from libraries now used > by the M3 runtime, but M3 is an open language and allows the integration > of C and C++ (and other) code (and thus of arbitrary other libraries). It is exactly what I was pointing to... Arbitrary other libraries will have defined entry points from Modula-3 code, and that is what matters... What and how C library code is doing with _its data_ is not of our concern. What is it doing with passed data concerns us, and it is a place where interfacing guidelines/policies kick in. ... > > Does it mean garbage collection is not > > incremental/generational/supercallifraglistic... anymore, or Anthony's > > work has made it work without memory protection? > > The latter; we've got incremental and generational garbage collection > without the need for memory protection and system call wrappers :) Oh. Oh. Ohhhhhhhhh. :) GREAT! > > Olaf -- Dragi?a Duri? From hosking at cs.purdue.edu Fri Apr 20 15:08:51 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:08:51 -0400 Subject: [M3devel] cm3 pthread problem? In-Reply-To: <1177071680.4282.47.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> Message-ID: I'm investigating this issue. Looks like some sort of race causing deadlock. On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: > Attached program won't run under freshly built cm3 @ Fedora Core 6. > Compiles, runs... and stucks... > > It works flawlessly with user-level threads and also with mine > implementation of NPTL for pm3. I've done it for 4000 threads per > 10000 > passess, and this version I am attaching, and trying without > success on > CM3 is 40 threads... > > Of course, before I run it, I am ulimiting stack size to 64kB or > similar.. It fits. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Fri Apr 20 15:16:33 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:16:33 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <1177056440.4282.34.camel@faramir.m3w.org> References: <1177056440.4282.34.camel@faramir.m3w.org> Message-ID: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> I have my hands full with GC evolution and thread issues at the moment. Anyone else care to look into this? On Apr 20, 2007, at 4:07 AM, Dragi?a Duri? wrote: > while trying to make my programs with newsly installed cm3, I am > constantly running into same error. Example: > > import("libm3") > > include_dir("../../../../m3lib0/src") > include_dir("../../../../pdf/src") > include_dir("../../../../zip/src") > > implementation("Test") > program("test") > > Is my m3makefile, and when "cm3" is run in package's src dir, I have > only: > > faramir:dragisha/pts/9: test/1/src% cm3 > --- building in ../LINUXLIBC6 --- > > > > *** > *** runtime error: > *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > list > *** file "../src/os/POSIX/PathnamePosix.m3", line 98 > *** > > Mentioned line raises this FATAL exception when Absolute(path) is > TRUE... What is absolute in my paths? > > All m3makefiles in question were ok with pm3, of course.. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at elegosoft.com Fri Apr 20 15:38:53 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 15:38:53 +0200 (CEST) Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <57472.194.138.127.36.1177076333.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 3:16 pm, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I can take this one, but won't be able to look into it until tonight (no M3 here at my current customer work), and if I don't find it fast, it will be delayed to Monday evening. If this is too late or anybody else can be faster, please let me know. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > >> while trying to make my programs with newsly installed cm3, I am >> constantly running into same error. Example: >> >> import("libm3") >> >> include_dir("../../../../m3lib0/src") >> include_dir("../../../../pdf/src") >> include_dir("../../../../zip/src") >> >> implementation("Test") >> program("test") >> >> Is my m3makefile, and when "cm3" is run in package's src dir, I have >> only: >> >> faramir:dragisha/pts/9: test/1/src% cm3 >> --- building in ../LINUXLIBC6 --- >> >> >> >> *** >> *** runtime error: >> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >> list >> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >> *** >> >> Mentioned line raises this FATAL exception when Absolute(path) is >> TRUE... What is absolute in my paths? >> >> All m3makefiles in question were ok with pm3, of course.. >> >> dd >> -- >> Dragi??a Duri?? >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 21:36:25 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 21:36:25 +0200 Subject: [M3devel] order of module initialization... Message-ID: <1177097786.4282.75.camel@faramir.m3w.org> I remember debugging this first time we tried to migrate to CM3... And then we solved it, for current CM3 and current version of our software, mentioning some imported module in EXPORTS clause... at the moment, this linked these modules in initialization chain and it worked ok... Right now, it does not. Module XLBuiltin is one imported regularly, and XLModuleUDP (for example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ wrong moment... What would be solution for this? dd -- Dragi?a Duri? From jayk123 at hotmail.com Fri Apr 20 23:58:55 2007 From: jayk123 at hotmail.com (j k) Date: Fri, 20 Apr 2007 21:58:55 +0000 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: Reminds me of C++ globals with constructors. The solution is to avoid initialization. Do everything "on demand" and hide all state behind functions. and then "on demand" needs to be thread safe, so there is a small bootstrapping problem -- basically the only initialization you should do, speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more accurately, nothing "in the same layer", if you have a dependency graph, only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are (unless you are working on ntdll.dll, but even then.. :) ) If you don't have a strict layered design, or can't capture what it is and analyze statically, again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an easy way to be correct. And even then, you can often avoid the critical sections via something like InterlockedCompareExchangePointer, even implement spin locks for this rare code. (Hand written spin locks are generally to be avoided since the kernel won't know about them and you won't schedule efficiently, whereas with EnterCriticalSection, if there is contention, the kernel has a hint what is going on and can schedule better). Don't have any "public global data", unless, and I don't know if this applies in Modula-3, unless it is "constant initialized", which I'm not sure is well defined in C++ or not. I do believe it is well defined in C..except dynamic linking I think screws that up to. (pardon the Win32-centricity; I'm sure the principles apply more broadly) Ok, in C++ and in Modula-3, there are maybe better answers. In C++, globals within a source file are guaranteed to be initialized in the order they occur in the file. But the order of files is not defined. There is a documented trick..potentially very inefficient..that I don't have the patience to describe here.. "the iostream counter trick"... this I don't think has an analog in Modula-3, but sure. Stroupstroup's Design and Evolution book documents it. - Jay >From: Dragi??a Duri?? >To: "M3 Developers' Mailing List" >Subject: [M3devel] order of module initialization... >Date: Fri, 20 Apr 2007 21:36:25 +0200 > >I remember debugging this first time we tried to migrate to CM3... And >then we solved it, for current CM3 and current version of our software, >mentioning some imported module in EXPORTS clause... at the moment, this >linked these modules in initialization chain and it worked ok... Right >now, it does not. > >Module XLBuiltin is one imported regularly, and XLModuleUDP (for >example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >wrong moment... > >What would be solution for this? > >dd >-- >Dragi??a Duri?? > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _________________________________________________________________ Download Messenger. Join the i?m Initiative. Help make a difference today. http://im.live.com/messenger/im/home/?source=TAGHM_APR07 From mika at async.caltech.edu Sat Apr 21 00:15:16 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 20 Apr 2007 15:15:16 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Fri, 20 Apr 2007 21:58:55 -0000." Message-ID: <200704202215.l3KMFGY4013501@camembert.async.caltech.edu> The order in Modula-3 is defined in section 2.5.6 of the language report (page 47 of Systems Programming with Modula-3). The problem described here appears to be that EXPORTS isn't treated the same as IMPORT (which it should be). There are some other problems I noticed when porting PM3 code to CM3, don't know if they have been fixed. The "solution" in that case was to IMPORT interfaces in places where they weren't really needed. Obfuscating one's code to avoid taking advantage of the convenience of module initialization seems like a rather severe suggestion... The rule, by the way, is: "If module M depends on module N and N does not depend on M, then N's body will be executed before M's body, where: * A module M 'depends on' a module N if M uses an interface that N exports or if M depends on a module that depends on N. * A module M 'uses' an interface I if M imports or exports I or if M uses an interface that imports I. Except for this constraint, the order of execution is implementation-dependent." Mika "j k" writes: >Reminds me of C++ globals with constructors. > >The solution is to avoid initialization. > >Do everything "on demand" and hide all state behind functions. > and then "on demand" needs to be thread safe, so there is a small >bootstrapping problem -- basically the only initialization you should do, >speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >accurately, nothing "in the same layer", if you have a dependency graph, >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are >(unless you are working on ntdll.dll, but even then.. :) ) If you don't have >a strict layered design, or can't capture what it is and analyze statically, >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an >easy way to be correct. > >And even then, you can often avoid the critical sections via something like >InterlockedCompareExchangePointer, even implement spin locks for this rare >code. (Hand written spin locks are generally to be avoided since the kernel >won't know about them and you won't schedule efficiently, whereas with >EnterCriticalSection, if there is contention, the kernel has a hint what is >going on and can schedule better). > >Don't have any "public global data", unless, and I don't know if this >applies in Modula-3, unless it is "constant initialized", which I'm not sure >is well defined in C++ or not. I do believe it is well defined in C..except >dynamic linking I think screws that up to. > >(pardon the Win32-centricity; I'm sure the principles apply more broadly) > >Ok, in C++ and in Modula-3, there are maybe better answers. >In C++, globals within a source file are guaranteed to be initialized in the >order they occur in the file. >But the order of files is not defined. >There is a documented trick..potentially very inefficient..that I don't have >the patience to describe here.. "the iostream counter trick"... this I don't >think has an analog in Modula-3, but sure. >Stroupstroup's Design and Evolution book documents it. > >- Jay > > >>From: Dragi??a Duri?? >>To: "M3 Developers' Mailing List" >>Subject: [M3devel] order of module initialization... >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >>I remember debugging this first time we tried to migrate to CM3... And >>then we solved it, for current CM3 and current version of our software, >>mentioning some imported module in EXPORTS clause... at the moment, this >>linked these modules in initialization chain and it worked ok... Right >>now, it does not. >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >>wrong moment... >> >>What would be solution for this? >> >>dd >>-- >>Dragi??a Duri?? >> >>_______________________________________________ >>M3devel mailing list >>M3devel at elegosoft.com >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Download Messenger. Join the i?m Initiative. Help make a difference today. >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Sun Apr 22 09:29:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 22 Apr 2007 09:29:11 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> Message-ID: <1177226951.737.1.camel@faramir.m3w.org> It looks like this fix never came through... With cvs-head CM3, unpickling of TextRefTbl.T instance still does not work. dd On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > Rodney, > > I've checked in fixes to RTTypeMap and clients that should restore > functional pickling support. For some reason my commit messages are > awaiting moderator approval (it seems my commit id is not the same as > my subscription to the m3commit list) before they can be sent out. > Please make sure that unpickling now works for you. > > Best regards, > > Tony > > On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > > > I tracked down a bug unpickling to the following. When executing > > in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > > > > WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > > ref^ := v.readRef(nelem.refType); > > END; > > > > has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > > CheckStoreTraced expects ref to be a normal reference to an object, > > computes the address of its header (4 bytes less), and sets the > > dirty bit. > > > > But ConvertPacking is reading in to a field of a recently created > > object that is not first in the object. It computes ref to point > > to the field, and CheckStoreTraced actually steps on the previous > > real field instead of the header. > > > > It looks like the actions of CheckStoreTraced are needed, but I > > don't right off hand see how to recode Convert to fix this. It's > > use of ref can't be moved. Presumably, there could be other unsafe > > code elsewhere using the same technique to store data. > > > > Does the compiler need a different criterion on when to generate > > the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > > still is pointing inside a traced object. Maybe people who modify > > objects in this way need to explicitly call CheckStoreTraced? > > > > -- > > ------------------------------------------------------------- > > Rodney M. Bates, retired assistant professor > > Dept. of Computer Science, Wichita State University > > Wichita, KS 67260-0083 > > 316-978-3922 > > rodney.bates at wichita.edu > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > Antony Hosking | Associate Professor > Dept of Computer Science | Office: +1 765 494-6001 > Purdue University | Mobile: +1 765 427-5484 > 250 N. University Street | Email: hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:22:11 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:22:11 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177226951.737.1.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: Yes, this fix is in CVS head. On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > It looks like this fix never came through... With cvs-head CM3, > unpickling of TextRefTbl.T instance still does not work. > > dd > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >> Rodney, >> >> I've checked in fixes to RTTypeMap and clients that should restore >> functional pickling support. For some reason my commit messages are >> awaiting moderator approval (it seems my commit id is not the same as >> my subscription to the m3commit list) before they can be sent out. >> Please make sure that unpickling now works for you. >> >> Best regards, >> >> Tony >> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >> >>> I tracked down a bug unpickling to the following. When executing >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>> >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>> ref^ := v.readRef(nelem.refType); >>> END; >>> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). >>> CheckStoreTraced expects ref to be a normal reference to an object, >>> computes the address of its header (4 bytes less), and sets the >>> dirty bit. >>> >>> But ConvertPacking is reading in to a field of a recently created >>> object that is not first in the object. It computes ref to point >>> to the field, and CheckStoreTraced actually steps on the previous >>> real field instead of the header. >>> >>> It looks like the actions of CheckStoreTraced are needed, but I >>> don't right off hand see how to recode Convert to fix this. It's >>> use of ref can't be moved. Presumably, there could be other unsafe >>> code elsewhere using the same technique to store data. >>> >>> Does the compiler need a different criterion on when to generate >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>> still is pointing inside a traced object. Maybe people who modify >>> objects in this way need to explicitly call CheckStoreTraced? >>> >>> -- >>> ------------------------------------------------------------- >>> Rodney M. Bates, retired assistant professor >>> Dept. of Computer Science, Wichita State University >>> Wichita, KS 67260-0083 >>> 316-978-3922 >>> rodney.bates at wichita.edu >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> Antony Hosking | Associate Professor >> Dept of Computer Science | Office: +1 765 494-6001 >> Purdue University | Mobile: +1 765 427-5484 >> 250 N. University Street | Email: hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:47:50 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:47:50 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177233124.737.4.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> Message-ID: <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> It is a pthreads call that is failing: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; I'm guessing we are in a race with a thread that is in the middle of dying (i.e., the error code, "r", is "ESRCH: thread is an invalid thread ID"). Can you confirm that is the error? As for why it is happening I don't know, since the thread handle is only ever accessed while activeMu is locked, which enforces atomicity of the thread's presence in the ring of live threads along with its handle being valid: WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; IF allThreads = me THEN allThreads := me.next; END; me.next.prev := me.prev; me.prev.next := me.next; me.next := NIL; me.prev := NIL; WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; I wonder if this is a system limit on the number of threads or some such (but then why would the pthread_create call not return an error?) Can you diagnose further? On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: > After upping it a bit, with stack limit set to 64k and 4000 > threads in > each loop... Is not happening at same place every run, but most of > runs > - it happens. > > Forking 4000 threads... (1) > Forked. > ,: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 > *** > > > On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >> I have fixed this bug and checked in the fix to CVS (why are we not >> seeing the commit messages again?). It did turn out to be a deadlock >> issue in the handshake between allocating threads and the GC. Your >> sample program now runs with no problems on my dual-core Intel Mac. >> Please let me know if you have any other threading difficulties. >> >> Regards, >> >> Tony >> >> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >> >>> Attached program won't run under freshly built cm3 @ Fedora Core 6. >>> Compiles, runs... and stucks... >>> >>> It works flawlessly with user-level threads and also with mine >>> implementation of NPTL for pm3. I've done it for 4000 threads per >>> 10000 >>> passess, and this version I am attaching, and trying without >>> success on >>> CM3 is 40 threads... >>> >>> Of course, before I run it, I am ulimiting stack size to 64kB or >>> similar.. It fits. >>> >>> dd >>> -- >>> Dragi?a Duri? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:59:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:59:46 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177253233.737.18.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> Message-ID: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > Hello, > > Have been skimming through source of PTHREAD code... And I think > job can > be done without so much relying on how-they-did-it-before, esp with > regard to list of waiters and similar internal and global structures. > Also, I see number of global locks and I am sure they are congestion > generators every now and while, esp in heavy threading situations. > > Of course, there is number of approaches to this multi-thread > situations. Mine being one of very nonconservative use of threads, I > think it is important to remain open to possibly very big number of > threads running in single process - meaning scalability is one of > primary objections... As global locks don't do well with > scalability, I > don't like "cm" and similar global congestion points. Yes, there are tensions between a thin/absent veneer between language threads and system threads. Most important are issues of preserving a reasonable memory model for programmers (see Hans Boehm's paper http://portal.acm.org/citation.cfm?doid=1065010.1065042). There are also questions of portability and debuggability. I agree that global locks are to be avoided where they cause known contention, but there are tradeoffs there too. For large numbers of threads (as you appear to need) I think we would need to adopt some other implementation approach, possibly by multiplexing multiple lightweight user-level threads on some smaller number of heavyweight system-level threads, but then you run into scheduling and load-balancing problems. > We talked about this at least once before, and I think I remember you > insisted on more compatibility than can be read from SPwM3. Do you > think > best idea would be to integrate mine NPTL code into CM3 for people to > trash and test, and let everyone select what is best for their > situation? What I wanted was a situation where programs would be able to run with the same tools (e.g., showheap, showthread) under both user- level and system threading. This goal has been achieved with the current pthread-based implementation. Moreover, I wanted something where variations in thread support from one system to another could be exploited most easily (such as for systems where thread suspend/ resume is provided as a primitive). Again, the current implementation achieves this, and runs with minimal target-specific code on Darwin, Solaris, and Linux. Ports to other targets should be relatively straightforward. > Problems I had with my pthread implementation were all related to VM > hell of earlier GC implementation... After you did that piece of art > with new approach to GC, I expect infinite uptimes from my servers and > bots :). Big thanks for that! Any native threading implementation is going to have problems with VM- based memory management. I'm surprised to were able to get anything going with the VM-based GC. > > dd > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 23:50:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 17:50:46 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177275494.737.37.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> <1177275494.737.37.camel@faramir.m3w.org> Message-ID: <8C54780A-5D27-4239-92BE-1D274340BE5A@cs.purdue.edu> Argh! That signal isn't even documented as being returned by pthread_kill on Linux (at least in my man pages). So the solution appears to be to replace: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; with: LOOP WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO IF r = 0 THEN EXIT; END; IF r # Uerror.EAGAIN THEN <*ASSERT FALSE*>; END; (* try it again *) END; END; What do you think? I wonder how many other places I should be checking for EAGAIN? On Apr 22, 2007, at 4:58 PM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:47 -0400, Tony Hosking wrote: >> It is a pthreads call that is failing: >> >> WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO >> <*ASSERT r=0*> >> END; >> >> I'm guessing we are in a race with a thread that is in the middle of >> dying (i.e., the error code, "r", is "ESRCH: thread is an invalid >> thread ID"). Can you confirm that is the error? > > No :). It's 11 - EAGAIN, IIRC. Lot's of signals are sent there, and it > kicked some signal dequeuing limit, or something like. > >> As for why it is happening I don't know, since the thread handle is >> only ever accessed while activeMu is locked, which enforces atomicity >> of the thread's presence in the ring of live threads along with its >> handle being valid: >> >> WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; >> IF allThreads = me THEN allThreads := me.next; END; >> me.next.prev := me.prev; >> me.prev.next := me.next; >> me.next := NIL; >> me.prev := NIL; >> WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; >> WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; >> >> I wonder if this is a system limit on the number of threads or some >> such (but then why would the pthread_create call not return an >> error?) Can you diagnose further? > > Limit is not, it's sure. I am running test in parallel with mine > implementation + pm3, and it works without a flaw, 10000 rounds, 4000 > threads each. I am using some realtime signal for suspend magic, > though... Don't remember right now, but I think there is something > special about this queuing of realtime and non-realtime signals... > >> >> On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: >> >>> After upping it a bit, with stack limit set to 64k and 4000 >>> threads in >>> each loop... Is not happening at same place every run, but most of >>> runs >>> - it happens. >>> >>> Forking 4000 threads... (1) >>> Forked. >>> ,: >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 >>> *** >>> >>> >>> On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >>>> I have fixed this bug and checked in the fix to CVS (why are we not >>>> seeing the commit messages again?). It did turn out to be a >>>> deadlock >>>> issue in the handshake between allocating threads and the GC. Your >>>> sample program now runs with no problems on my dual-core Intel Mac. >>>> Please let me know if you have any other threading difficulties. >>>> >>>> Regards, >>>> >>>> Tony >>>> >>>> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >>>> >>>>> Attached program won't run under freshly built cm3 @ Fedora >>>>> Core 6. >>>>> Compiles, runs... and stucks... >>>>> >>>>> It works flawlessly with user-level threads and also with mine >>>>> implementation of NPTL for pm3. I've done it for 4000 threads per >>>>> 10000 >>>>> passess, and this version I am attaching, and trying without >>>>> success on >>>>> CM3 is 40 threads... >>>>> >>>>> Of course, before I run it, I am ulimiting stack size to 64kB or >>>>> similar.. It fits. >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 00:00:03 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:00:03 -0400 Subject: [M3devel] Fwd: A GC/compiler/Pickle interaction bug References: <1177277134.737.47.camel@faramir.m3w.org> Message-ID: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> This works just fine for me on the most recent CVS head for I386_DARWIN. I wonder if you have not updated your compiler to match? There was a matching fix to the compiler. You should build and ship: m3gc-simple m3core libm3 m3middle m3linker m3front m3quake cm3 Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your cm3.cfg). This will get you a new compiler. Then, rebuild your CM3 installation from the CVS head. Begin forwarded message: > From: Dragi?a Duri? > Date: April 22, 2007 5:25:33 PM EDT > To: Tony Hosking > Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug > > On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >> Can you send me a stack dump from gdb at the point of the error? >> Even better, do you have a simple example program I can use to test >> with? > > Here is one. Quick and dirty, but it shows problem. > > -- > Dragi?a Duri? -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Pickles.m3 URL: -------------- next part -------------- From hosking at cs.purdue.edu Mon Apr 23 00:01:05 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:01:05 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> Message-ID: <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> PS The bootstrap compilers on elegosoft probably may to be rebuilt for this fix to take effect. On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: > This works just fine for me on the most recent CVS head for > I386_DARWIN. > > I wonder if you have not updated your compiler to match? There was > a matching fix to the compiler. You should build and ship: > > m3gc-simple > m3core > libm3 > m3middle > m3linker > m3front > m3quake > cm3 > > Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your > cm3.cfg). This will get you a new compiler. > > Then, rebuild your CM3 installation from the CVS head. > > Begin forwarded message: > >> From: Dragi?a Duri? >> Date: April 22, 2007 5:25:33 PM EDT >> To: Tony Hosking >> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >> >> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>> Can you send me a stack dump from gdb at the point of the error? >>> Even better, do you have a simple example program I can use to test >>> with? >> >> Here is one. Quick and dirty, but it shows problem. >> >> -- >> Dragi?a Duri? >> > From dragisha at m3w.org Mon Apr 23 10:21:16 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 23 Apr 2007 10:21:16 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> Message-ID: <1177316476.737.97.camel@faramir.m3w.org> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > > > Hello, > > > > Have been skimming through source of PTHREAD code... And I think > > job can > > be done without so much relying on how-they-did-it-before, esp with > > regard to list of waiters and similar internal and global structures. > > Also, I see number of global locks and I am sure they are congestion > > generators every now and while, esp in heavy threading situations. > > > > Of course, there is number of approaches to this multi-thread > > situations. Mine being one of very nonconservative use of threads, I > > think it is important to remain open to possibly very big number of > > threads running in single process - meaning scalability is one of > > primary objections... As global locks don't do well with > > scalability, I > > don't like "cm" and similar global congestion points. > > Yes, there are tensions between a thin/absent veneer between language > threads and system threads. Most important are issues of preserving > a reasonable memory model for programmers (see Hans Boehm's paper > http://portal.acm.org/citation.cfm?doid=1065010.1065042). I know that paper, and being Modula-3 camp, I am - by definition - agreed to no-library-approach-for-threads :). > There are > also questions of portability and debuggability. Of course. That's why I am using only POSIX defined features, and when in doubt - ones used by Boehm in his famous GC :). > I agree that global > locks are to be avoided where they cause known contention, but there > are tradeoffs there too. Global lock is bad, whatever reasons. > For large numbers of threads (as you appear > to need) I think we would need to adopt some other implementation > approach, possibly by multiplexing multiple lightweight user-level > threads on some smaller number of heavyweight system-level threads, > but then you run into scheduling and load-balancing problems. I've argued this before... With O(1) process scheduling available for four years from Linux, and in that time surely from everyone else... It would be bigger problem to maintain scheduling for special "BIG" cases inside our support libraries than to rely on operating system. It's good that mainstream OS people recognized threads as Need, and it is time now for us to accept they did it well - AT LAST. And, very important... I can't see what is heavyweight on system which does 10,000 context switches per second for 1500 threads with 2% CPU load. And all this in 2004 on some 1.something GHz CPU. Threads WERE heavyweight four years ago, and they are not, long since. Even Windows has lightweight kernel-space threads :). > > > We talked about this at least once before, and I think I remember you > > insisted on more compatibility than can be read from SPwM3. Do you > > think > > best idea would be to integrate mine NPTL code into CM3 for people to > > trash and test, and let everyone select what is best for their > > situation? > > What I wanted was a situation where programs would be able to run > with the same tools (e.g., showheap, showthread) under both user- > level and system threading. This goal has been achieved with the > current pthread-based implementation. It is good reasoning, and it's one of reasons I did not suggest replacement... I think mine version is less bloated and I know it's very good for long and stable process uptime we all expect from Modula-3 programs. But also, implied compatibilities outside of SPwM3 and direct demands from other parts of runtime were not on my list. These I've respected, and it looks like these are good production time criteria. As opposed to excellent development time criteria you based yours on. > Moreover, I wanted something > where variations in thread support from one system to another could > be exploited most easily (such as for systems where thread suspend/ > resume is provided as a primitive). Again, the current > implementation achieves this, and runs with minimal target-specific > code on Darwin, Solaris, and Linux. Ports to other targets should be > relatively straightforward. I've not ported mine outside of LINUXLIBC6, but as it's extremmely POSIX, I see no problem. > > > Problems I had with my pthread implementation were all related to VM > > hell of earlier GC implementation... After you did that piece of art > > with new approach to GC, I expect infinite uptimes from my servers and > > bots :). Big thanks for that! > > Any native threading implementation is going to have problems with VM- > based memory management. I'm surprised to were able to get anything > going with the VM-based GC. Anything is pretty much - I have heavy multithreaded servers running literally for years,,, one of them is up since January of 2004, it services few hundreds of connected users (and up to 1500 threads) at almost every moment and breaks only when system reboots :). All that with heavy integration of various C libraries. > > > > > dd > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 14:40:44 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 23 Apr 2007 08:40:44 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177316476.737.97.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: I take all of your points seriously. One option would be to offer your threads implementation as another build option for CM3. I'm going to track down the bug I introduced recently and then we can consider how to move forward. On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >> >>> Hello, >>> >>> Have been skimming through source of PTHREAD code... And I think >>> job can >>> be done without so much relying on how-they-did-it-before, esp with >>> regard to list of waiters and similar internal and global >>> structures. >>> Also, I see number of global locks and I am sure they are congestion >>> generators every now and while, esp in heavy threading situations. >>> >>> Of course, there is number of approaches to this multi-thread >>> situations. Mine being one of very nonconservative use of threads, I >>> think it is important to remain open to possibly very big number of >>> threads running in single process - meaning scalability is one of >>> primary objections... As global locks don't do well with >>> scalability, I >>> don't like "cm" and similar global congestion points. >> >> Yes, there are tensions between a thin/absent veneer between language >> threads and system threads. Most important are issues of preserving >> a reasonable memory model for programmers (see Hans Boehm's paper >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > I know that paper, and being Modula-3 camp, I am - by definition - > agreed to no-library-approach-for-threads :). > >> There are >> also questions of portability and debuggability. > > Of course. That's why I am using only POSIX defined features, and > when > in doubt - ones used by Boehm in his famous GC :). > >> I agree that global >> locks are to be avoided where they cause known contention, but there >> are tradeoffs there too. > > Global lock is bad, whatever reasons. > >> For large numbers of threads (as you appear >> to need) I think we would need to adopt some other implementation >> approach, possibly by multiplexing multiple lightweight user-level >> threads on some smaller number of heavyweight system-level threads, >> but then you run into scheduling and load-balancing problems. > > I've argued this before... With O(1) process scheduling available > for > four years from Linux, and in that time surely from everyone > else... It > would be bigger problem to maintain scheduling for special "BIG" cases > inside our support libraries than to rely on operating system. It's > good > that mainstream OS people recognized threads as Need, and it is > time now > for us to accept they did it well - AT LAST. > > And, very important... I can't see what is heavyweight on system > which > does 10,000 context switches per second for 1500 threads with 2% CPU > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > heavyweight four years ago, and they are not, long since. Even Windows > has lightweight kernel-space threads :). > >> >>> We talked about this at least once before, and I think I remember >>> you >>> insisted on more compatibility than can be read from SPwM3. Do you >>> think >>> best idea would be to integrate mine NPTL code into CM3 for >>> people to >>> trash and test, and let everyone select what is best for their >>> situation? >> >> What I wanted was a situation where programs would be able to run >> with the same tools (e.g., showheap, showthread) under both user- >> level and system threading. This goal has been achieved with the >> current pthread-based implementation. > > It is good reasoning, and it's one of reasons I did not suggest > replacement... I think mine version is less bloated and I know it's > very > good for long and stable process uptime we all expect from Modula-3 > programs. But also, implied compatibilities outside of SPwM3 and > direct > demands from other parts of runtime were not on my list. These I've > respected, and it looks like these are good production time > criteria. As > opposed to excellent development time criteria you based yours on. > >> Moreover, I wanted something >> where variations in thread support from one system to another could >> be exploited most easily (such as for systems where thread suspend/ >> resume is provided as a primitive). Again, the current >> implementation achieves this, and runs with minimal target-specific >> code on Darwin, Solaris, and Linux. Ports to other targets should be >> relatively straightforward. > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > POSIX, I see no problem. > >> >>> Problems I had with my pthread implementation were all related to VM >>> hell of earlier GC implementation... After you did that piece of art >>> with new approach to GC, I expect infinite uptimes from my >>> servers and >>> bots :). Big thanks for that! >> >> Any native threading implementation is going to have problems with >> VM- >> based memory management. I'm surprised to were able to get anything >> going with the VM-based GC. > > Anything is pretty much - I have heavy multithreaded servers running > literally for years,,, one of them is up since January of 2004, it > services few hundreds of connected users (and up to 1500 threads) at > almost every moment and breaks only when system reboots :). All that > with heavy integration of various C libraries. > >> >>> >>> dd >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From mika at async.caltech.edu Tue Apr 24 22:21:25 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 24 Apr 2007 13:21:25 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Tue, 24 Apr 2007 14:15:19 -0000." Message-ID: <200704242021.l3OKLPac036051@camembert.async.caltech.edu> Yes, that is the entire rule. There can definitely be cycles in the graph; in this case, the rule "if M depends on N and N does not depend on M" does not apply, as they depend on each other. The order of initialization is then implementation-dependent. Just creating objects of type (circularly) implemented by another module doesn't present any special problems, unless the correct functioning of those objects is dependent on module-initialization code. And I don't know about your compiler, but mine won't link things that didn't compile. I have to admit that the module initialization rule does break a nice property of Modula-3 programs, which is that you can't normally break a module by making a change to another module. If your module M indirectly depends on module N and furthermore, the correct functioning of the M-N relationship depends on module N's being initialized before module M, the addition in module N (in private implementation code) of a dependency on module M's exported interface can cause your program to fail. Mika "j k" writes: >ok. Can there be circularities in the graph? >Two modules creating objects of types implemented by the other in their >module initializer? Or that won't compile? I'll have to try it. > >- Jay > > >>From: Mika Nystrom >>To: "j k" >>CC: dragisha at m3w.org, m3devel at elegosoft.com >>Subject: Re: [M3devel] order of module initialization... Date: Fri, 20 Apr >>2007 15:15:16 -0700 >> >>The order in Modula-3 is defined in section 2.5.6 of the language >>report (page 47 of Systems Programming with Modula-3). >> >>The problem described here appears to be that EXPORTS isn't treated >>the same as IMPORT (which it should be). There are some other >>problems I noticed when porting PM3 code to CM3, don't know if they >>have been fixed. The "solution" in that case was to IMPORT interfaces >>in places where they weren't really needed. >> >>Obfuscating one's code to avoid taking advantage of the convenience >>of module initialization seems like a rather severe suggestion... >> >>The rule, by the way, is: >> >>"If module M depends on module N and N does not depend on M, then >>N's body will be executed before M's body, where: >> >>* A module M 'depends on' a module N if M uses an interface that N >>exports or if M depends on a module that depends on N. >> >>* A module M 'uses' an interface I if M imports or exports I or if >>M uses an interface that imports I. >> >>Except for this constraint, the order of execution is >>implementation-dependent." >> >> Mika >> >>"j k" writes: >> >Reminds me of C++ globals with constructors. >> > >> >The solution is to avoid initialization. >> > >> >Do everything "on demand" and hide all state behind functions. >> > and then "on demand" needs to be thread safe, so there is a small >> >bootstrapping problem -- basically the only initialization you should do, >> >speaking in Win32-ese, is a few calls to InitializeCriticalSection and >>maybe >> >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >> >accurately, nothing "in the same layer", if you have a dependency graph, >> >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always >>are >> >(unless you are working on ntdll.dll, but even then.. :) ) If you don't >>have >> >a strict layered design, or can't capture what it is and analyze >>statically, >> >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is >>an >> >easy way to be correct. >> > >> >And even then, you can often avoid the critical sections via something >>like >> >InterlockedCompareExchangePointer, even implement spin locks for this >rare >> >code. (Hand written spin locks are generally to be avoided since the >>kernel >> >won't know about them and you won't schedule efficiently, whereas with >> >EnterCriticalSection, if there is contention, the kernel has a hint what >>is >> >going on and can schedule better). >> > >> >Don't have any "public global data", unless, and I don't know if this >> >applies in Modula-3, unless it is "constant initialized", which I'm not >>sure >> >is well defined in C++ or not. I do believe it is well defined in >>C..except >> >dynamic linking I think screws that up to. >> > >> >(pardon the Win32-centricity; I'm sure the principles apply more broadly) >> > >> >Ok, in C++ and in Modula-3, there are maybe better answers. >> >In C++, globals within a source file are guaranteed to be initialized in >>the >> >order they occur in the file. >> >But the order of files is not defined. >> >There is a documented trick..potentially very inefficient..that I don't >>have >> >the patience to describe here.. "the iostream counter trick"... this I >>don't >> >think has an analog in Modula-3, but sure. >> >Stroupstroup's Design and Evolution book documents it. >> > >> >- Jay >> > >> > >> >>From: Dragi??a Duri?? >> >>To: "M3 Developers' Mailing List" >> >>Subject: [M3devel] order of module initialization... >> >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >> >> >>I remember debugging this first time we tried to migrate to CM3... And >> >>then we solved it, for current CM3 and current version of our software, >> >>mentioning some imported module in EXPORTS clause... at the moment, this >> >>linked these modules in initialization chain and it worked ok... Right >> >>now, it does not. >> >> >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >> >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >> >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >> >>wrong moment... >> >> >> >>What would be solution for this? >> >> >> >>dd >> >>-- >> >>Dragi??a Duri?? >> >> >> >>_______________________________________________ >> >>M3devel mailing list >> >>M3devel at elegosoft.com >> >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >_________________________________________________________________ >> >Download Messenger. Join the i?m Initiative. Help make a difference >>today. >> >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 >> > >> >_______________________________________________ >> >M3devel mailing list >> >M3devel at elegosoft.com >> >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Mortgage rates near historic lows. Refinance $200,000 loan for as low as >$771/month* >https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&search=m >ortgage_text_links_88_h27f8&disc=y&vers=689&s=4056&p=5117 From dragisha at m3w.org Wed Apr 25 08:55:12 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 08:55:12 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: <1177484112.4826.9.camel@faramir.m3w.org> Just a random thought... I don't think my TestThreads is something special, but it's few thread use patterns combined... And I've just had bright :) idea yesterday - it's also decent benchmark for whole threading system... I think it would be nice to test it with 10000 rounds, 4000 threads each (once cm3 cvs-head is fixed) with PTHREAD and without PTHREAD. I will do tests for mine. I think these extra data structures and global locks in PTHREAD are big efficiency killers. Benchmark will show. dd On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > I take all of your points seriously. One option would be to offer > your threads implementation as another build option for CM3. I'm > going to track down the bug I introduced recently and then we can > consider how to move forward. > > On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > > > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >> > >>> Hello, > >>> > >>> Have been skimming through source of PTHREAD code... And I think > >>> job can > >>> be done without so much relying on how-they-did-it-before, esp with > >>> regard to list of waiters and similar internal and global > >>> structures. > >>> Also, I see number of global locks and I am sure they are congestion > >>> generators every now and while, esp in heavy threading situations. > >>> > >>> Of course, there is number of approaches to this multi-thread > >>> situations. Mine being one of very nonconservative use of threads, I > >>> think it is important to remain open to possibly very big number of > >>> threads running in single process - meaning scalability is one of > >>> primary objections... As global locks don't do well with > >>> scalability, I > >>> don't like "cm" and similar global congestion points. > >> > >> Yes, there are tensions between a thin/absent veneer between language > >> threads and system threads. Most important are issues of preserving > >> a reasonable memory model for programmers (see Hans Boehm's paper > >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > > > I know that paper, and being Modula-3 camp, I am - by definition - > > agreed to no-library-approach-for-threads :). > > > >> There are > >> also questions of portability and debuggability. > > > > Of course. That's why I am using only POSIX defined features, and > > when > > in doubt - ones used by Boehm in his famous GC :). > > > >> I agree that global > >> locks are to be avoided where they cause known contention, but there > >> are tradeoffs there too. > > > > Global lock is bad, whatever reasons. > > > >> For large numbers of threads (as you appear > >> to need) I think we would need to adopt some other implementation > >> approach, possibly by multiplexing multiple lightweight user-level > >> threads on some smaller number of heavyweight system-level threads, > >> but then you run into scheduling and load-balancing problems. > > > > I've argued this before... With O(1) process scheduling available > > for > > four years from Linux, and in that time surely from everyone > > else... It > > would be bigger problem to maintain scheduling for special "BIG" cases > > inside our support libraries than to rely on operating system. It's > > good > > that mainstream OS people recognized threads as Need, and it is > > time now > > for us to accept they did it well - AT LAST. > > > > And, very important... I can't see what is heavyweight on system > > which > > does 10,000 context switches per second for 1500 threads with 2% CPU > > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > > heavyweight four years ago, and they are not, long since. Even Windows > > has lightweight kernel-space threads :). > > > >> > >>> We talked about this at least once before, and I think I remember > >>> you > >>> insisted on more compatibility than can be read from SPwM3. Do you > >>> think > >>> best idea would be to integrate mine NPTL code into CM3 for > >>> people to > >>> trash and test, and let everyone select what is best for their > >>> situation? > >> > >> What I wanted was a situation where programs would be able to run > >> with the same tools (e.g., showheap, showthread) under both user- > >> level and system threading. This goal has been achieved with the > >> current pthread-based implementation. > > > > It is good reasoning, and it's one of reasons I did not suggest > > replacement... I think mine version is less bloated and I know it's > > very > > good for long and stable process uptime we all expect from Modula-3 > > programs. But also, implied compatibilities outside of SPwM3 and > > direct > > demands from other parts of runtime were not on my list. These I've > > respected, and it looks like these are good production time > > criteria. As > > opposed to excellent development time criteria you based yours on. > > > >> Moreover, I wanted something > >> where variations in thread support from one system to another could > >> be exploited most easily (such as for systems where thread suspend/ > >> resume is provided as a primitive). Again, the current > >> implementation achieves this, and runs with minimal target-specific > >> code on Darwin, Solaris, and Linux. Ports to other targets should be > >> relatively straightforward. > > > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > > POSIX, I see no problem. > > > >> > >>> Problems I had with my pthread implementation were all related to VM > >>> hell of earlier GC implementation... After you did that piece of art > >>> with new approach to GC, I expect infinite uptimes from my > >>> servers and > >>> bots :). Big thanks for that! > >> > >> Any native threading implementation is going to have problems with > >> VM- > >> based memory management. I'm surprised to were able to get anything > >> going with the VM-based GC. > > > > Anything is pretty much - I have heavy multithreaded servers running > > literally for years,,, one of them is up since January of 2004, it > > services few hundreds of connected users (and up to 1500 threads) at > > almost every moment and breaks only when system reboots :). All that > > with heavy integration of various C libraries. > > > >> > >>> > >>> dd > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From wagner at elegosoft.com Wed Apr 25 09:10:20 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:10:20 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: <462EFEDC.5090000@elegosoft.com> Dragi?a Duri? wrote: > I remember debugging this first time we tried to migrate to CM3... And > then we solved it, for current CM3 and current version of our software, > mentioning some imported module in EXPORTS clause... at the moment, this > linked these modules in initialization chain and it worked ok... Right > now, it does not. > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > wrong moment... > > What would be solution for this? > The order of module initialization in CM3 is extensively traced. Try to run your program with @M3tracelinker and see what's wrong. The code for module initialization is in cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 If the initialization does not work as expected, we must be missing of overlooking some dependency there. If you can provide a simple and small example that does not work, I can probably have a look at it at the next weekend. Sorry for the late answer, Olaf From dragisha at m3w.org Wed Apr 25 09:44:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 09:44:52 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <462EFEDC.5090000@elegosoft.com> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> Message-ID: <1177487092.4826.15.camel@faramir.m3w.org> I've debugged this before, and I know the drill... Problem with initialization, sometimes, is in it passing pass 2 of RTLinker for some modules and never coming to pass 3... I've noticed that. Last few days I am using -linkall, and after I've deleted some redundant EXPORTS it kind of works ok. I remember you telling me about this non-initialization of non-imported modules once before. It is where CM3 differs from both SRC and PM3 in big way. Don't remember rationalization, but I don't like situation where I can't make/exploit plugin architecture simply by mentioning new module in m3makefile and letting it initialize itself in standardized manner. dd On Wed, 2007-04-25 at 09:10 +0200, Olaf Wagner wrote: > Dragi?a Duri? wrote: > > I remember debugging this first time we tried to migrate to CM3... And > > then we solved it, for current CM3 and current version of our software, > > mentioning some imported module in EXPORTS clause... at the moment, this > > linked these modules in initialization chain and it worked ok... Right > > now, it does not. > > > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > > wrong moment... > > > > What would be solution for this? > > > The order of module initialization in CM3 is extensively traced. Try to > run your program with @M3tracelinker and see what's wrong. The > code for module initialization is in > > cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 > > If the initialization does not work as expected, we must be missing > of overlooking some dependency there. > > If you can provide a simple and small example that does not work, > I can probably have a look at it at the next weekend. > > Sorry for the late answer, > > Olaf -- Dragi?a Duri? From wagner at mx0.elegosoft.com Wed Apr 25 09:00:22 2007 From: wagner at mx0.elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:00:22 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <20070425070022.GA4827@elegosoft.com> On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I've checked in a fix for this monday evening, thought the commit message again wasn't delivered. I also instructed Ronny to have a look at the commit message setup again. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > > >while trying to make my programs with newsly installed cm3, I am > >constantly running into same error. Example: > > > >import("libm3") > > > >include_dir("../../../../m3lib0/src") > >include_dir("../../../../pdf/src") > >include_dir("../../../../zip/src") > > > >implementation("Test") > >program("test") > > > >Is my m3makefile, and when "cm3" is run in package's src dir, I have > >only: > > > >faramir:dragisha/pts/9: test/1/src% cm3 > >--- building in ../LINUXLIBC6 --- > > > > > > > >*** > >*** runtime error: > >*** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > >list > >*** file "../src/os/POSIX/PathnamePosix.m3", line 98 > >*** > > > >Mentioned line raises this FATAL exception when Absolute(path) is > >TRUE... What is absolute in my paths? > > > >All m3makefiles in question were ok with pm3, of course.. > > > >dd > >-- > >Dragi??a Duri?? > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Wed Apr 25 11:33:44 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 11:33:44 +0200 (CEST) Subject: [M3devel] order of module initialization... In-Reply-To: <1177487092.4826.15.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> <1177487092.4826.15.camel@faramir.m3w.org> Message-ID: <62295.194.138.127.36.1177493624.squirrel@mail.elegosoft.com> On Wed, April 25, 2007 9:44 am, Dragi??a Duri?� wrote: > I've debugged this before, and I know the drill... Problem with > initialization, sometimes, is in it passing pass 2 of RTLinker for some > modules and never coming to pass 3... I've noticed that. Last few days I > am using -linkall, and after I've deleted some redundant EXPORTS it kind > of works ok. > > I remember you telling me about this non-initialization of non-imported > modules once before. It is where CM3 differs from both SRC and PM3 in > big way. Don't remember rationalization, but I don't like situation > where I can't make/exploit plugin architecture simply by mentioning new > module in m3makefile and letting it initialize itself in standardized > manner. I'm afraid I still don't understand the problem well enough though I think we've probably got a bug somewhere in the initialization code. It would be really helpful if you could provide a simple example for debugging. I'll take care of the problem then (though it may take some time). Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 25 15:51:52 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:51:52 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177484112.4826.9.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: Yes, good thinking. Tuning the threads systems is a good plan all round. On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > Just a random thought... > > I don't think my TestThreads is something special, but it's few thread > use patterns combined... And I've just had bright :) idea yesterday - > it's also decent benchmark for whole threading system... I think it > would be nice to test it with 10000 rounds, 4000 threads each (once > cm3 > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > tests for > mine. > > I think these extra data structures and global locks in PTHREAD are > big > efficiency killers. Benchmark will show. > > dd > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >> I take all of your points seriously. One option would be to offer >> your threads implementation as another build option for CM3. I'm >> going to track down the bug I introduced recently and then we can >> consider how to move forward. >> >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >> >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>> >>>>> Hello, >>>>> >>>>> Have been skimming through source of PTHREAD code... And I think >>>>> job can >>>>> be done without so much relying on how-they-did-it-before, esp >>>>> with >>>>> regard to list of waiters and similar internal and global >>>>> structures. >>>>> Also, I see number of global locks and I am sure they are >>>>> congestion >>>>> generators every now and while, esp in heavy threading situations. >>>>> >>>>> Of course, there is number of approaches to this multi-thread >>>>> situations. Mine being one of very nonconservative use of >>>>> threads, I >>>>> think it is important to remain open to possibly very big >>>>> number of >>>>> threads running in single process - meaning scalability is one of >>>>> primary objections... As global locks don't do well with >>>>> scalability, I >>>>> don't like "cm" and similar global congestion points. >>>> >>>> Yes, there are tensions between a thin/absent veneer between >>>> language >>>> threads and system threads. Most important are issues of >>>> preserving >>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>> >>> I know that paper, and being Modula-3 camp, I am - by definition - >>> agreed to no-library-approach-for-threads :). >>> >>>> There are >>>> also questions of portability and debuggability. >>> >>> Of course. That's why I am using only POSIX defined features, and >>> when >>> in doubt - ones used by Boehm in his famous GC :). >>> >>>> I agree that global >>>> locks are to be avoided where they cause known contention, but >>>> there >>>> are tradeoffs there too. >>> >>> Global lock is bad, whatever reasons. >>> >>>> For large numbers of threads (as you appear >>>> to need) I think we would need to adopt some other implementation >>>> approach, possibly by multiplexing multiple lightweight user-level >>>> threads on some smaller number of heavyweight system-level threads, >>>> but then you run into scheduling and load-balancing problems. >>> >>> I've argued this before... With O(1) process scheduling available >>> for >>> four years from Linux, and in that time surely from everyone >>> else... It >>> would be bigger problem to maintain scheduling for special "BIG" >>> cases >>> inside our support libraries than to rely on operating system. It's >>> good >>> that mainstream OS people recognized threads as Need, and it is >>> time now >>> for us to accept they did it well - AT LAST. >>> >>> And, very important... I can't see what is heavyweight on system >>> which >>> does 10,000 context switches per second for 1500 threads with 2% CPU >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE >>> heavyweight four years ago, and they are not, long since. Even >>> Windows >>> has lightweight kernel-space threads :). >>> >>>> >>>>> We talked about this at least once before, and I think I remember >>>>> you >>>>> insisted on more compatibility than can be read from SPwM3. Do you >>>>> think >>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>> people to >>>>> trash and test, and let everyone select what is best for their >>>>> situation? >>>> >>>> What I wanted was a situation where programs would be able to run >>>> with the same tools (e.g., showheap, showthread) under both user- >>>> level and system threading. This goal has been achieved with the >>>> current pthread-based implementation. >>> >>> It is good reasoning, and it's one of reasons I did not suggest >>> replacement... I think mine version is less bloated and I know it's >>> very >>> good for long and stable process uptime we all expect from Modula-3 >>> programs. But also, implied compatibilities outside of SPwM3 and >>> direct >>> demands from other parts of runtime were not on my list. These I've >>> respected, and it looks like these are good production time >>> criteria. As >>> opposed to excellent development time criteria you based yours on. >>> >>>> Moreover, I wanted something >>>> where variations in thread support from one system to another could >>>> be exploited most easily (such as for systems where thread suspend/ >>>> resume is provided as a primitive). Again, the current >>>> implementation achieves this, and runs with minimal target-specific >>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>> should be >>>> relatively straightforward. >>> >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely >>> POSIX, I see no problem. >>> >>>> >>>>> Problems I had with my pthread implementation were all related >>>>> to VM >>>>> hell of earlier GC implementation... After you did that piece >>>>> of art >>>>> with new approach to GC, I expect infinite uptimes from my >>>>> servers and >>>>> bots :). Big thanks for that! >>>> >>>> Any native threading implementation is going to have problems with >>>> VM- >>>> based memory management. I'm surprised to were able to get >>>> anything >>>> going with the VM-based GC. >>> >>> Anything is pretty much - I have heavy multithreaded servers >>> running >>> literally for years,,, one of them is up since January of 2004, it >>> services few hundreds of connected users (and up to 1500 threads) at >>> almost every moment and breaks only when system reboots :). All that >>> with heavy integration of various C libraries. >>> >>>> >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed Apr 25 15:52:49 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:52:49 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <20070425070022.GA4827@elegosoft.com> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> <20070425070022.GA4827@elegosoft.com> Message-ID: PS CVS head is broken right now due to my most recent "fixes" to threading. I am currently looking into the problem. On Apr 25, 2007, at 3:00 AM, Olaf Wagner wrote: > On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: >> I have my hands full with GC evolution and thread issues at the >> moment. Anyone else care to look into this? > > I've checked in a fix for this monday evening, thought the commit > message again wasn't delivered. I also instructed Ronny to have > a look at the commit message setup again. > > Olaf > >> On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: >> >>> while trying to make my programs with newsly installed cm3, I am >>> constantly running into same error. Example: >>> >>> import("libm3") >>> >>> include_dir("../../../../m3lib0/src") >>> include_dir("../../../../pdf/src") >>> include_dir("../../../../zip/src") >>> >>> implementation("Test") >>> program("test") >>> >>> Is my m3makefile, and when "cm3" is run in package's src dir, I have >>> only: >>> >>> faramir:dragisha/pts/9: test/1/src% cm3 >>> --- building in ../LINUXLIBC6 --- >>> >>> >>> >>> *** >>> *** runtime error: >>> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >>> list >>> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >>> *** >>> >>> Mentioned line raises this FATAL exception when Absolute(path) is >>> TRUE... What is absolute in my paths? >>> >>> All m3makefiles in question were ok with pm3, of course.. >>> >>> dd >>> -- >>> Dragi??a Duri?? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Wed Apr 25 18:35:37 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 18:35:37 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: <1177518937.15060.6.camel@faramir.m3w.org> After implementing that workaround for result code 11 in that SIGSUSPEND loop, every time during 1st or at most second pass (of 4000) it stucks. Not same place every time, though... I think there are RCs in LockHeap/SuspendAll logic. dd On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > Yes, good thinking. Tuning the threads systems is a good plan all > round. > > On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > > > Just a random thought... > > > > I don't think my TestThreads is something special, but it's few thread > > use patterns combined... And I've just had bright :) idea yesterday - > > it's also decent benchmark for whole threading system... I think it > > would be nice to test it with 10000 rounds, 4000 threads each (once > > cm3 > > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > > tests for > > mine. > > > > I think these extra data structures and global locks in PTHREAD are > > big > > efficiency killers. Benchmark will show. > > > > dd > > > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >> I take all of your points seriously. One option would be to offer > >> your threads implementation as another build option for CM3. I'm > >> going to track down the bug I introduced recently and then we can > >> consider how to move forward. > >> > >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >> > >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>> > >>>>> Hello, > >>>>> > >>>>> Have been skimming through source of PTHREAD code... And I think > >>>>> job can > >>>>> be done without so much relying on how-they-did-it-before, esp > >>>>> with > >>>>> regard to list of waiters and similar internal and global > >>>>> structures. > >>>>> Also, I see number of global locks and I am sure they are > >>>>> congestion > >>>>> generators every now and while, esp in heavy threading situations. > >>>>> > >>>>> Of course, there is number of approaches to this multi-thread > >>>>> situations. Mine being one of very nonconservative use of > >>>>> threads, I > >>>>> think it is important to remain open to possibly very big > >>>>> number of > >>>>> threads running in single process - meaning scalability is one of > >>>>> primary objections... As global locks don't do well with > >>>>> scalability, I > >>>>> don't like "cm" and similar global congestion points. > >>>> > >>>> Yes, there are tensions between a thin/absent veneer between > >>>> language > >>>> threads and system threads. Most important are issues of > >>>> preserving > >>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>> > >>> I know that paper, and being Modula-3 camp, I am - by definition - > >>> agreed to no-library-approach-for-threads :). > >>> > >>>> There are > >>>> also questions of portability and debuggability. > >>> > >>> Of course. That's why I am using only POSIX defined features, and > >>> when > >>> in doubt - ones used by Boehm in his famous GC :). > >>> > >>>> I agree that global > >>>> locks are to be avoided where they cause known contention, but > >>>> there > >>>> are tradeoffs there too. > >>> > >>> Global lock is bad, whatever reasons. > >>> > >>>> For large numbers of threads (as you appear > >>>> to need) I think we would need to adopt some other implementation > >>>> approach, possibly by multiplexing multiple lightweight user-level > >>>> threads on some smaller number of heavyweight system-level threads, > >>>> but then you run into scheduling and load-balancing problems. > >>> > >>> I've argued this before... With O(1) process scheduling available > >>> for > >>> four years from Linux, and in that time surely from everyone > >>> else... It > >>> would be bigger problem to maintain scheduling for special "BIG" > >>> cases > >>> inside our support libraries than to rely on operating system. It's > >>> good > >>> that mainstream OS people recognized threads as Need, and it is > >>> time now > >>> for us to accept they did it well - AT LAST. > >>> > >>> And, very important... I can't see what is heavyweight on system > >>> which > >>> does 10,000 context switches per second for 1500 threads with 2% CPU > >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > >>> heavyweight four years ago, and they are not, long since. Even > >>> Windows > >>> has lightweight kernel-space threads :). > >>> > >>>> > >>>>> We talked about this at least once before, and I think I remember > >>>>> you > >>>>> insisted on more compatibility than can be read from SPwM3. Do you > >>>>> think > >>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>> people to > >>>>> trash and test, and let everyone select what is best for their > >>>>> situation? > >>>> > >>>> What I wanted was a situation where programs would be able to run > >>>> with the same tools (e.g., showheap, showthread) under both user- > >>>> level and system threading. This goal has been achieved with the > >>>> current pthread-based implementation. > >>> > >>> It is good reasoning, and it's one of reasons I did not suggest > >>> replacement... I think mine version is less bloated and I know it's > >>> very > >>> good for long and stable process uptime we all expect from Modula-3 > >>> programs. But also, implied compatibilities outside of SPwM3 and > >>> direct > >>> demands from other parts of runtime were not on my list. These I've > >>> respected, and it looks like these are good production time > >>> criteria. As > >>> opposed to excellent development time criteria you based yours on. > >>> > >>>> Moreover, I wanted something > >>>> where variations in thread support from one system to another could > >>>> be exploited most easily (such as for systems where thread suspend/ > >>>> resume is provided as a primitive). Again, the current > >>>> implementation achieves this, and runs with minimal target-specific > >>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>> should be > >>>> relatively straightforward. > >>> > >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely > >>> POSIX, I see no problem. > >>> > >>>> > >>>>> Problems I had with my pthread implementation were all related > >>>>> to VM > >>>>> hell of earlier GC implementation... After you did that piece > >>>>> of art > >>>>> with new approach to GC, I expect infinite uptimes from my > >>>>> servers and > >>>>> bots :). Big thanks for that! > >>>> > >>>> Any native threading implementation is going to have problems with > >>>> VM- > >>>> based memory management. I'm surprised to were able to get > >>>> anything > >>>> going with the VM-based GC. > >>> > >>> Anything is pretty much - I have heavy multithreaded servers > >>> running > >>> literally for years,,, one of them is up since January of 2004, it > >>> services few hundreds of connected users (and up to 1500 threads) at > >>> almost every moment and breaks only when system reboots :). All that > >>> with heavy integration of various C libraries. > >>> > >>>> > >>>>> > >>>>> dd > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From darko at darko.org Tue Apr 3 15:33:23 2007 From: darko at darko.org (Darko) Date: Tue, 3 Apr 2007 15:33:23 +0200 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> Message-ID: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> I'm not sure if this has been resolved in some more proper way, but for the record, the warnings appearing after installation of the XCode 2.4.1 tools can be gotten rid of by changing the following in the config file: proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-o", object) end so it reads proc assemble (source, object) is return try_exec ("@/usr/bin/as", source, "-W -o", object) end It's pretty obvious, but just in case anyone goes searching these archives for an answer... On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: > >> >> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >> >>> For some reason your .stabs entries are different than mine, but >>> otherwise the assembler files are the same. Are you using the >>> cm3cg and cm3.cfg from my ftp site? >> >> Yes, downloaded a few hours ago. >> >> >>> Here is the version of the assembler that I have on my Intel Mac: >>> >>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>> assembler version 1.38 >> >> The mine is: >> >> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >> version 1.38 >> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >> > > Hmmm -- different versions of the assembler. Probably some flag > needs adjusting. > >> >> >>> >>> With respect to zeus, can you build with -verbose in the zeus >>> directory and send me the output? Sounds like one of the tools >>> is not getting built properly. >>> >> >> Ok, I am sending it as attachment >> >> >> > > Sorry, wrong option. Please run with -commands. I suspect stubgen > is failing for you. > > >> Renzo >> >> >>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>> >>>> You can find in the attachment the program and the compilation. >>>> >>>> Renzo >>>> >>>> >>>> >>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>> >>>>> >>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>> >>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>> >>>>>>> >>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>> >>>>>>>> Hello and lot of thanks! >>>>>>>> >>>>>>>> Everything is working really fine! >>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>> anonymous CVS), >>>>>>>> The process: >>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>> 3. running as su: >>>>>>>> do-cm3-core.sh buildship >>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>> do-cm3-std.sh buildship >>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>> and failing at the end the compilation of package zeus >>>>>>>> (which I suppose is an old issue) >>>>>>> >>>>>>> That's odd since my builds go through cleanly. >>>>>> >>>>>> This is an example of the strange "indirect jump" message: >>>>>> >>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>> --- building in I386_DARWIN --- >>>>>> new source -> compiling Main.m3 >>>>>> Main.ms:101:indirect jmp without `*' >>>>>> -> linking prog >>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>> Hello World >>>>>> >>>>>> maybe there is some problem with the linker parameters, >>>>>> however the progrum runs ok. >>>>> >>>>> Can you compile with "-keep" and send me the relevant .ms >>>>> file? I can compare with mine. >>>>> >>>>>> >>>>>> For what concerns zeus: the following is the output of cm3: >>>>>> >>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>> cvsstuff/cm3' +++ >>>>>> --- building in I386_DARWIN --- >>>>>> >>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>> missing INTERFACE or MODULE keyword >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>> interface () >>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>> id>) >>>>>> 2 errors and 1 warning encountered >>>>>> compilation failed => not building library "libm3zeus.a" >>>>>> Fatal Error: package build failed >>>>>> *** execution of failed *** >>>>>> >>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>> >>>>>> Several other packages cannot be compiled, like many obliq >>>>>> ones, webvt, June and mentor. >>>>>> >>>>>> However the compiler seems to work for compiling my stuff, >>>>>> altough I have to make a few corrections and debugging. >>>>> >>>>> This sounds problematic since it implies that the generators >>>>> for those files are not working correctly. (If you run with "- >>>>> commands" you'll see which generator programs are being used. >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Renzo >>>>>> >>>>>>> >>>>>>>> >>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>> century! >>>>>>>> >>>>>>>> Cordially, >>>>>>>> >>>>>>>> Renzo Orsini >>>>>>>> >>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>> >>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>> about 1G, by the way. >>>>>>>>> >>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>> >>>>>>>>>> What version of OSX are you using? >>>>>>>>>> >>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>> >>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>> >>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>> >>>>>>>>>>> - Darko >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>> >>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>> >>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/local/ >>>>>>>>>>>> cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>> this is the result: >>>>>>>>>>>> >>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>> CM3C = >>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh -c >>>>>>>>>>>> "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector >>>>>>>>>>>> digraph parseparams realgeometry set slisp >>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: pointer >>>>>>>>>>>> targets in passing argument 3 of 'accept' differ in >>>>>>>>>>>> signedness >>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>> differ in signedness >>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>> >>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>> errno=2 >>>>>>>>>>>> >>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>> install_file -- >>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>> >>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>> >>>>>>>>>>>> ====== >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I do >>>>>>>>>>>> not know if this is a normal behaviour. >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>> >>>>>>>>>>>> Cordially >>>>>>>>>>>> >>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>> >>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>> >>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>> >>>>>>>>>>>>> Darko. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo Orsini >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on an >>>>>>>>>>>>>>> intel mac, then >>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Since I would like to port the Fibonacci language for >>>>>>>>>>>>>>> databases, >>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>> documentation to start with. Of course I will be very >>>>>>>>>>>>>>> glad to give >>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>> >>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>> to install >>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>> be too >>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>> on it, >>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>> mail to >>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>> such an >>>>>>>>>>>>>> effort. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>> m3devel >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> M3devel mailing list >>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>> >>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>> ~hosking >>>>>>>>>> _--_|\ >>>>>>>>>> / \ >>>>>>>>>> \_.--._/ ) >>>>>>>>>> v / >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Antony Hosking | Associate Professor >>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>> ~hosking >>>>>>> _--_|\ >>>>>>> / \ >>>>>>> \_.--._/ ) >>>>>>> v / >>>>>>> >>>>>> >>>>> >>>>> Antony Hosking | Associate Professor >>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>> Purdue University | Mobile: (765) 427-5484 >>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>> _--_|\ >>>>> / \ >>>>> \_.--._/ ) >>>>> v / >>>>> >>>> >>> >>> Antony Hosking | Associate Professor >>> Dept of Computer Sciences | Office: (765) 494-6001 >>> Purdue University | Mobile: (765) 427-5484 >>> 250 N. University Street | hosking at cs.purdue.edu >>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>> _--_|\ >>> / \ >>> \_.--._/ ) >>> v / >>> >> > > Antony Hosking | Associate Professor > Dept of Computer Sciences | Office: (765) 494-6001 > Purdue University | Mobile: (765) 427-5484 > 250 N. University Street | hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Tue Apr 3 16:31:13 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 3 Apr 2007 10:31:13 -0400 Subject: [M3devel] Re: porting m3 on intel mac In-Reply-To: <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> References: <8A58B35B-22CD-42D5-BA19-5FBF5D3CF5CD@dsi.unive.it> <20070120164128.GA27790@elegosoft.com> <52EA8FC9-4E33-40FF-B041-5CEAE83BBDE7@darko.org> <6307E3CC-8E5F-4317-8A8E-7EADE20FEE54@darko.org> <1521F74A-AF0F-48C4-B0AA-4E7DB321DA62@cs.purdue.edu> <23B90155-784C-4A47-8EAD-1451AEC4C1B9@darko.org> <69645CDA-E215-4B81-BAEE-C45EC496E4C6@dsi.unive.it> <5C13EB47-D241-438A-BE6F-BBD30AF002A4@dsi.unive.it> <9125FF3E-5D5D-4DE0-95DC-DCD997D198C1@cs.purdue.edu> <4206B526-A4CF-430B-AE8C-086D27615A79@dsi.unive.it> <722DDD36-2973-44B2-91BF-8487DA744D6C@cs.purdue.edu> <6607A270-8264-4419-AAF2-323833523695@cs.purdue.edu> <58EDFF76-646E-4140-B382-6B831301FE3D@darko.org> Message-ID: <9753494C-96FE-4BDE-81AE-8CDDE9BC398B@cs.purdue.edu> -q also works. On Apr 3, 2007, at 9:33 AM, Darko wrote: > I'm not sure if this has been resolved in some more proper way, but > for the record, the warnings appearing after installation of the > XCode 2.4.1 tools can be gotten rid of by changing the following in > the config file: > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-o", object) > end > > so it reads > > proc assemble (source, object) is > return try_exec ("@/usr/bin/as", source, "-W -o", object) > end > > It's pretty obvious, but just in case anyone goes searching these > archives for an answer... > > > On 21/01/2007, at 11:32 PM, Antony Hosking wrote: > >> On 21/01/2007, at 5:06 PM, Renzo Orsini wrote: >> >>> >>> On Jan 21, 2007, at 22:57, Antony Hosking wrote: >>> >>>> For some reason your .stabs entries are different than mine, but >>>> otherwise the assembler files are the same. Are you using the >>>> cm3cg and cm3.cfg from my ftp site? >>> >>> Yes, downloaded a few hours ago. >>> >>> >>>> Here is the version of the assembler that I have on my Intel Mac: >>>> >>>> Apple Computer, Inc. version cctools-590.42.1.obj~1, GNU >>>> assembler version 1.38 >>> >>> The mine is: >>> >>> Apple Computer, Inc. version cctools-622.5.obj~13, GNU assembler >>> version 1.38 >>> (If it useful, my Xcode is 2.4.1, with Xcode IDE: 762.0, Xcode >>> Core: 762.0, ToolSupport: 764.0, [From About XCode menu]) >>> >> >> Hmmm -- different versions of the assembler. Probably some flag >> needs adjusting. >> >>> >>> >>>> >>>> With respect to zeus, can you build with -verbose in the zeus >>>> directory and send me the output? Sounds like one of the tools >>>> is not getting built properly. >>>> >>> >>> Ok, I am sending it as attachment >>> >>> >>> >> >> Sorry, wrong option. Please run with -commands. I suspect >> stubgen is failing for you. >> >> >>> Renzo >>> >>> >>>> On 21/01/2007, at 4:47 PM, Renzo Orsini wrote: >>>> >>>>> You can find in the attachment the program and the compilation. >>>>> >>>>> Renzo >>>>> >>>>> >>>>> >>>>> On Jan 21, 2007, at 22:34, Antony Hosking wrote: >>>>> >>>>>> >>>>>> On 21/01/2007, at 4:27 PM, Renzo Orsini wrote: >>>>>> >>>>>>> On Jan 21, 2007, at 21:49, Antony Hosking wrote: >>>>>>> >>>>>>>> >>>>>>>> On 21/01/2007, at 7:59 AM, Renzo Orsini wrote: >>>>>>>> >>>>>>>>> Hello and lot of thanks! >>>>>>>>> >>>>>>>>> Everything is working really fine! >>>>>>>>> The configuration: MacBookPro, MacOSX 10.4.8, XCode 2.4.1 >>>>>>>>> (gcc 4.0.1), X11 installed, latest cm3 sources (with >>>>>>>>> anonymous CVS), >>>>>>>>> The process: >>>>>>>>> 1. unpacked cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz >>>>>>>>> 2. installed as super-user (every answer left as default, >>>>>>>>> the only problem is the fact that motif is missing, ignoring) >>>>>>>>> 3. replacing cm3, cm3.cfg, cm3cg from ftp:// >>>>>>>>> ftp.cs.purdue.edu/pub/hosking/m3/I386_DARWIN >>>>>>>>> 3. running as su: >>>>>>>>> do-cm3-core.sh buildship >>>>>>>>> install-cm3-compiler.sh upgrade >>>>>>>>> do-cm3-std.sh buildship >>>>>>>>> everything is ok a part from producing about a zillion of >>>>>>>>> assembler (?) warnings ("xxx.ms:nnn:indirect jmp without `*'), >>>>>>>>> and failing at the end the compilation of package zeus >>>>>>>>> (which I suppose is an old issue) >>>>>>>> >>>>>>>> That's odd since my builds go through cleanly. >>>>>>> >>>>>>> This is an example of the strange "indirect jump" message: >>>>>>> >>>>>>> pborsini:~/ProveModula3 orsini$ cat Main.m3 >>>>>>> MODULE Main; IMPORT IO; BEGIN IO.Put("Hello World\n"); END Main. >>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>> --- building in I386_DARWIN --- >>>>>>> new source -> compiling Main.m3 >>>>>>> Main.ms:101:indirect jmp without `*' >>>>>>> -> linking prog >>>>>>> _m3main.ms:74:indirect jmp without `*' >>>>>>> _m3main.ms:89:indirect jmp without `*' >>>>>>> _m3main.ms:108:indirect jmp without `*' >>>>>>> pborsini:~/ProveModula3 orsini$ ./I386_DARWIN/prog >>>>>>> Hello World >>>>>>> >>>>>>> maybe there is some problem with the linker parameters, >>>>>>> however the progrum runs ok. >>>>>> >>>>>> Can you compile with "-keep" and send me the relevant .ms >>>>>> file? I can compare with mine. >>>>>> >>>>>>> >>>>>>> For what concerns zeus: the following is the output of cm3: >>>>>>> >>>>>>> === package /Users/orsini/modula3/cvsstuff/cm3/m3-ui/zeus === >>>>>>> +++ cm3 -build -override -DROOT='/Users/orsini/modula3/ >>>>>>> cvsstuff/cm3' +++ >>>>>>> --- building in I386_DARWIN --- >>>>>>> >>>>>>> new source -> compiling RemoteView_T_v1.i3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.i3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.i3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> new source -> compiling RemoteView_T_v1.m3 >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: syntax error: >>>>>>> missing INTERFACE or MODULE keyword >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: unable to find >>>>>>> interface () >>>>>>> "../I386_DARWIN/RemoteView_T_v1.m3", line 1: warning: file >>>>>>> name (RemoteView_T_v1.m3) doesn't match module name (>>>>>> id>) >>>>>>> 2 errors and 1 warning encountered >>>>>>> compilation failed => not building library "libm3zeus.a" >>>>>>> Fatal Error: package build failed >>>>>>> *** execution of failed *** >>>>>>> >>>>>>> (and the files RemoteView_T_v1.i3 and RemoteView_T_v1.m3 are >>>>>>> empty in I386_DARWIN, and they do not exists in src) >>>>>>> >>>>>>> Several other packages cannot be compiled, like many obliq >>>>>>> ones, webvt, June and mentor. >>>>>>> >>>>>>> However the compiler seems to work for compiling my stuff, >>>>>>> altough I have to make a few corrections and debugging. >>>>>> >>>>>> This sounds problematic since it implies that the generators >>>>>> for those files are not working correctly. (If you run with "- >>>>>> commands" you'll see which generator programs are being used. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Renzo >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> So, thank again to all and each of you, I am now very happy >>>>>>>>> and busy by bringing our old modula-3 software to the new >>>>>>>>> century! >>>>>>>>> >>>>>>>>> Cordially, >>>>>>>>> >>>>>>>>> Renzo Orsini >>>>>>>>> >>>>>>>>> On Jan 21, 2007, at 0:33, Darko wrote: >>>>>>>>> >>>>>>>>>> The latest, 10.4.8, also the latest XCode, which I think >>>>>>>>>> updates gcc. You were addressing me weren't you? But Renzo >>>>>>>>>> might like to answer that question too. XCode weighs in at >>>>>>>>>> about 1G, by the way. >>>>>>>>>> >>>>>>>>>> On 21/01/2007, at 7:58 AM, Antony Hosking wrote: >>>>>>>>>> >>>>>>>>>>> What version of OSX are you using? >>>>>>>>>>> >>>>>>>>>>> On 20/01/2007, at 5:49 PM, Darko wrote: >>>>>>>>>>> >>>>>>>>>>>> No worries. I'm not familiar at all with those scripts. >>>>>>>>>>>> The problem you are having may be to do with not having >>>>>>>>>>>> the latest source, but it most probably has to do with >>>>>>>>>>>> the version of gcc that is active on your machine. C >>>>>>>>>>>> files are not compiled by cm3 but rather call gcc >>>>>>>>>>>> directly. I can't remember which version of gcc is the >>>>>>>>>>>> correct one, but mine currently is i686-apple-darwin8- >>>>>>>>>>>> gcc-4.0.1; also I can't remember the command line for >>>>>>>>>>>> changing gcc versions. Tony may be able to help re the >>>>>>>>>>>> correct version if 4.0.1 doesn't work. You may need to >>>>>>>>>>>> install the latest XCode from http://developer.apple.com >>>>>>>>>>>> if you don't have 4.01. >>>>>>>>>>>> >>>>>>>>>>>> I've cc'd the list because they're actually very helpful >>>>>>>>>>>> and my knowledge is limited... >>>>>>>>>>>> >>>>>>>>>>>> - Darko >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 21/01/2007, at 6:58 AM, Renzo Orsini wrote: >>>>>>>>>>>> >>>>>>>>>>>>> First of all, thank you very much for your immediate help! >>>>>>>>>>>>> >>>>>>>>>>>>> I downloaded the files and replaced the corresponding / >>>>>>>>>>>>> usr/local/cm3/bin files, >>>>>>>>>>>>> modified cm3.cfg by replacing INSTALL_ROOT = "/usr/ >>>>>>>>>>>>> local/cm3-i386/" with INSTALL_ROOT = "/usr/local/cm3/" >>>>>>>>>>>>> then in the full source (last version, 5.4.0) I did: >>>>>>>>>>>>> scripts/do-cm3-core.sh buildship >>>>>>>>>>>>> this is the result: >>>>>>>>>>>>> >>>>>>>>>>>>> root#./do-cm3-core.sh buildship >>>>>>>>>>>>> CM3C = >>>>>>>>>>>>> /Users/orsini/cm3/cm3-src-all-5.4.0/scripts/pkgmap.sh - >>>>>>>>>>>>> c "cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' " m3gc-simple m3core libm3 >>>>>>>>>>>>> patternmatching m3middle m3linker m3front m3quake m3cc >>>>>>>>>>>>> cm3 m3scanner m3tools m3cgcat m3cggen m3bundle >>>>>>>>>>>>> bitvector digraph parseparams realgeometry set slisp >>>>>>>>>>>>> sortedtableextras table-list tempfiles >>>>>>>>>>>>> === package /Users/orsini/cm3/cm3-src-all-5.4.0/m3-libs/ >>>>>>>>>>>>> m3gc-simple === >>>>>>>>>>>>> +++ cm3 -build -DROOT='/Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0' && cm3 -ship -DROOT='/Users/orsini/cm3/cm3- >>>>>>>>>>>>> src-all-5.4.0' +++ >>>>>>>>>>>>> --- building in I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> new source -> compiling RTVM.c >>>>>>>>>>>>> new source -> compiling sysdeps.c >>>>>>>>>>>>> new source -> compiling accept.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c: In function >>>>>>>>>>>>> 'm3_accept': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/accept.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'accept' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling bind.c >>>>>>>>>>>>> new source -> compiling close.c >>>>>>>>>>>>> new source -> compiling connect.c >>>>>>>>>>>>> new source -> compiling dup.c >>>>>>>>>>>>> new source -> compiling dup2.c >>>>>>>>>>>>> new source -> compiling gethostbyaddr.c >>>>>>>>>>>>> new source -> compiling gethostbyname.c >>>>>>>>>>>>> new source -> compiling getpeername.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c: In function >>>>>>>>>>>>> 'm3_getpeername': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getpeername.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getpeername' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling getsockname.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c: In function >>>>>>>>>>>>> 'm3_getsockname': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/getsockname.c:13: warning: >>>>>>>>>>>>> pointer targets in passing argument 3 of 'getsockname' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling listen.c >>>>>>>>>>>>> new source -> compiling read.c >>>>>>>>>>>>> new source -> compiling recv.c >>>>>>>>>>>>> new source -> compiling recvfrom.c >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c: In function >>>>>>>>>>>>> 'm3_recvfrom': >>>>>>>>>>>>> ../src/runtime/I386_DARWIN/recvfrom.c:15: warning: >>>>>>>>>>>>> pointer targets in passing argument 6 of 'recvfrom' >>>>>>>>>>>>> differ in signedness >>>>>>>>>>>>> new source -> compiling select.c >>>>>>>>>>>>> new source -> compiling send.c >>>>>>>>>>>>> new source -> compiling sendto.c >>>>>>>>>>>>> new source -> compiling shutdown.c >>>>>>>>>>>>> new source -> compiling socket.c >>>>>>>>>>>>> new source -> compiling write.c >>>>>>>>>>>>> -> archiving libm3gcdefs.a >>>>>>>>>>>>> ld: common symbols not allowed with MH_DYLIB output >>>>>>>>>>>>> format with the -multi_module option >>>>>>>>>>>>> sysdeps.o definition of common _RTCSRC_FinishVM (size 16) >>>>>>>>>>>>> sysdeps.o definition of common _RTHeapRep_Fault (size 16) >>>>>>>>>>>>> /usr/bin/libtool: internal link edit command failed >>>>>>>>>>>>> --- shipping from I386_DARWIN --- >>>>>>>>>>>>> >>>>>>>>>>>>> . => /usr/local/cm3/pkg/m3gc-simple/I386_DARWIN >>>>>>>>>>>>> .M3EXPORTS libm3gcdefs.5.2.dylib"/Users/orsini/ >>>>>>>>>>>>> cm3/cm3-src-all-5.4.0/m3-libs/m3gc-simple/ >>>>>>>>>>>>> I386_DARWIN/.M3SHIP", line 3: quake runtime error: >>>>>>>>>>>>> unable to copy "libm3gcdefs.5.2.dylib" to "/usr/local/ >>>>>>>>>>>>> cm3/pkg/m3gc-simple/I386_DARWIN/libm3gcdefs.5.2.dylib": >>>>>>>>>>>>> errno=2 >>>>>>>>>>>>> >>>>>>>>>>>>> --procedure-- -line- -file--- >>>>>>>>>>>>> install_file -- >>>>>>>>>>>>> 3 /Users/orsini/cm3/cm3-src- >>>>>>>>>>>>> all-5.4.0/m3-libs/m3gc-simple/I386_DARWIN/.M3SHIP >>>>>>>>>>>>> >>>>>>>>>>>>> Fatal Error: package build failed >>>>>>>>>>>>> *** execution of failed *** >>>>>>>>>>>>> >>>>>>>>>>>>> ====== >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I noticed that not all the .c files in .../m3-libs/m3gc- >>>>>>>>>>>>> simple/src/runtime/I386_DARWIN/ where compiled, but I >>>>>>>>>>>>> do not know if this is a normal behaviour. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks in advance for your time and patience! >>>>>>>>>>>>> >>>>>>>>>>>>> Cordially >>>>>>>>>>>>> >>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Jan 20, 2007, at 19:02, Darko wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Have a look here: ftp://ftp.cs.purdue.edu/pub/hosking/ >>>>>>>>>>>>>> m3/I386_DARWIN/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> There you'll find the bits you need for Mac Intel. >>>>>>>>>>>>>> Replace your existing cm3, cmcg and cm3.cfg with the >>>>>>>>>>>>>> ones there, then you can compile the basic libraries >>>>>>>>>>>>>> you need like m3core and libm3, but make sure you have >>>>>>>>>>>>>> the latest sources. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think they're the latest but Tony may have something >>>>>>>>>>>>>> more to say about that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Darko. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 21/01/2007, at 1:41 AM, Olaf Wagner wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Jan 20, 2007 at 03:05:55PM +0100, Renzo >>>>>>>>>>>>>>> Orsini wrote: >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I installed cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz on >>>>>>>>>>>>>>>> an intel mac, then >>>>>>>>>>>>>>>> tried to a compile a Hello world program, but the >>>>>>>>>>>>>>>> compilation failed >>>>>>>>>>>>>>>> with the following message: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> pborsini:~/ProveModula3 orsini$ cm3 >>>>>>>>>>>>>>>> --- building in PPC_DARWIN --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> new source -> compiling Main.m3 >>>>>>>>>>>>>>>> "../Main.m3", line 4: warning: potentially unhandled >>>>>>>>>>>>>>>> exception: IO.Error >>>>>>>>>>>>>>>> 1 warning encountered >>>>>>>>>>>>>>>> Main.ms:12:no such instruction: `mflr r0' >>>>>>>>>>>>>>>> Main.ms:13:no such instruction: `stmw r30,-8(r1)' >>>>>>>>>>>>>>>> Main.ms:14:no such instruction: `stw r0,8(r1)' >>>>>>>>>>>>>>>> Main.ms:15:no such instruction: `stwu r1,-96(r1)' >>>>>>>>>>>>>>>> Main.ms:16:no such instruction: `mr r30,r1' >>>>>>>>>>>>>>>> Main.ms:17:no such instruction: `bcl >>>>>>>>>>>>>>>> 20,31,"L00000000001$pb"' >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So I suppose there is PPC code to assemble and the >>>>>>>>>>>>>>>> Rosetta emulator >>>>>>>>>>>>>>>> for PPC on intel macs cannot do anything for this! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Since I would like to port the Fibonacci language >>>>>>>>>>>>>>>> for databases, >>>>>>>>>>>>>>>> which is written in Modula-3, if I understand well >>>>>>>>>>>>>>>> the situation I >>>>>>>>>>>>>>>> should port the Modula-3 compiler by cross compiling >>>>>>>>>>>>>>>> it on a PPC mac, >>>>>>>>>>>>>>>> or something like that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am asking you if you know about some effort of >>>>>>>>>>>>>>>> porting Modula-3 on >>>>>>>>>>>>>>>> intel macs, or at least if you let me know if this >>>>>>>>>>>>>>>> operation is >>>>>>>>>>>>>>>> possible (and also reasonably feasible in terms of >>>>>>>>>>>>>>>> time and >>>>>>>>>>>>>>>> expertise, given my teaching duties and my >>>>>>>>>>>>>>>> difficulties with >>>>>>>>>>>>>>>> assembler languages!), and, if so, if you can point >>>>>>>>>>>>>>>> me to the correct >>>>>>>>>>>>>>>> documentation to start with. Of course I will be >>>>>>>>>>>>>>>> very glad to give >>>>>>>>>>>>>>>> back to the community the result if I succeed! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you very much for your attention. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cordially >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Renzo Orsini >>>>>>>>>>>>>>>> Associate Professor >>>>>>>>>>>>>>>> Dipartimento di Informatica >>>>>>>>>>>>>>>> Universita' Ca' Foscari di Venezia >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Well, I wouldn't have thought that you were even able >>>>>>>>>>>>>>> to install >>>>>>>>>>>>>>> the cm3-min-POSIX-PPC_DARWIN-5.4.0.tgz system on an >>>>>>>>>>>>>>> Intel machine; >>>>>>>>>>>>>>> it's surely not supposed to run on one. It shouldn't >>>>>>>>>>>>>>> be too >>>>>>>>>>>>>>> difficult to get a port of the latest CM3 for Darwin >>>>>>>>>>>>>>> on Intel >>>>>>>>>>>>>>> processors; I even think somebody was already working >>>>>>>>>>>>>>> on it, >>>>>>>>>>>>>>> but I don't really remember who, so I forward your >>>>>>>>>>>>>>> mail to >>>>>>>>>>>>>>> the m3devel list, in case others have already started >>>>>>>>>>>>>>> such an >>>>>>>>>>>>>>> effort. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Olaf >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> elego Software Solutions >>>>>>>>>>>>>>> GmbH HRB 77719 >>>>>>>>>>>>>>> Olaf Wagner E-Mail: wagner >>>>>>>>>>>>>>> (at)elego.de >>>>>>>>>>>>>>> Ohmstra?e 9 Tel: +49 30 >>>>>>>>>>>>>>> 40 04 19 29 >>>>>>>>>>>>>>> 10179 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 23 45 86 95 >>>>>>>>>>>>>>> Cranachstra?e 7 Tel: +49 30 >>>>>>>>>>>>>>> 85 58 01 81 >>>>>>>>>>>>>>> 12157 Berlin Fax: +49 30 >>>>>>>>>>>>>>> 85 58 01 88 >>>>>>>>>>>>>>> ------------------> WWW: http://www.elego-software- >>>>>>>>>>>>>>> solutions.com >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/ >>>>>>>>>>>>>>> m3devel >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> M3devel mailing list >>>>>>>>>>>> M3devel at elegosoft.com >>>>>>>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>>>>>>> >>>>>>>>>>> Antony Hosking | Associate Professor >>>>>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>>>>> ~hosking >>>>>>>>>>> _--_|\ >>>>>>>>>>> / \ >>>>>>>>>>> \_.--._/ ) >>>>>>>>>>> v / >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Antony Hosking | Associate Professor >>>>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/ >>>>>>>> ~hosking >>>>>>>> _--_|\ >>>>>>>> / \ >>>>>>>> \_.--._/ ) >>>>>>>> v / >>>>>>>> >>>>>>> >>>>>> >>>>>> Antony Hosking | Associate Professor >>>>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>>>> Purdue University | Mobile: (765) 427-5484 >>>>>> 250 N. University Street | hosking at cs.purdue.edu >>>>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>>>> _--_|\ >>>>>> / \ >>>>>> \_.--._/ ) >>>>>> v / >>>>>> >>>>> >>>> >>>> Antony Hosking | Associate Professor >>>> Dept of Computer Sciences | Office: (765) 494-6001 >>>> Purdue University | Mobile: (765) 427-5484 >>>> 250 N. University Street | hosking at cs.purdue.edu >>>> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >>>> _--_|\ >>>> / \ >>>> \_.--._/ ) >>>> v / >>>> >>> >> >> Antony Hosking | Associate Professor >> Dept of Computer Sciences | Office: (765) 494-6001 >> Purdue University | Mobile: (765) 427-5484 >> 250 N. University Street | hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From stsp at elego.de Mon Apr 9 14:21:31 2007 From: stsp at elego.de (Stefan Sperling) Date: Mon, 9 Apr 2007 14:21:31 +0200 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461A2D05.6040208@elego.de> References: <46192AF7.6060406@elego.de> <20070408194650.GB32688@jack.stsp.lan> <4619836C.6060707@elego.de> <461A2D05.6040208@elego.de> Message-ID: <20070409122131.GB1350@ted.stsp.lan> On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels Janosch Hofmeyr wrote: > Just to complete the information in this thread: > ubuntu 6.10 does not have an asm/ipc.h. However, asm/ipc.h is not > necessary. It is sufficient to delete or comment the lines 19-21 in file > m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > #include > #endif > > This change has been committed to the repository, but was accidentally > not merged into the 5.4.0 release, apparently. Shall I create new source archives for the 5.4 branch that include this fix? If so, what version number should I put on them? 5.4.1? It should not be necessary to update the binary packages as well. Should I just copy them to a new name with the updated version number? -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From dabenavidesd at yahoo.es Mon Apr 9 21:47:35 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 9 Apr 2007 21:47:35 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409122131.GB1350@ted.stsp.lan> Message-ID: <636229.78700.qm@web27107.mail.ukl.yahoo.com> --- Stefan Sperling wrote: >On Mon, Apr 09, 2007 at 02:09:41PM +0200, Neels >Janosch Hofmeyr wrote: > Just to complete the information in this thread: > >It is sufficient to delete or comment > >the lines 19-21 in file > > > >m3-libs/m3gc-simple/src/runtime/LINUXLIBC6/sysdeps.c: > > > > #if __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 1 > > #include > > #endif > > > > This change has been committed to the repository, > but was accidentally > > not merged into the 5.4.0 release, apparently. The problem was reported on the m3devel list on february 6, so It couldnt be merged on cm3-src-all-5.4.0.tgz file if that is the proeblem, that you refer. > Shall I create new source archives for the 5.4 > branch that include this > fix? If so, what version number should I put on > them? 5.4.1? I think this is the most appropiate to put a new file to download with the sources updated (not just that sysdeps.c update). > It should not be necessary to update the binary > packages as well. > Should I just copy them to a new name with the > updated version number? Antony, the bootstrap for the Pthread version is different from the bootstrap of cm3-5.4.0 release? I have exectuted do-cm3-std.sh buildship with the current cvs sources with cm3-5.4.0 bootstrap and in just one package I get a problem; Juno-2 with ubuntu 6.10, when tries to do a redirection of the of the PklFonts executable. ./PklFonts > FontData.pkl It got segmentation fault The thing is, after the rest of the system is compiled the Juno-2 compiles with no problems, including that redirection. ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From dabenavidesd at yahoo.es Tue Apr 10 01:49:25 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 01:49:25 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: Message-ID: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Hello: I have updated the libfreetype6 (to libfreetype6 2.2.1-5) library in ubuntu 6.10, and its now corrected the problem on juno with PklFonts, because, I have executed do-cm3-std.sh buildship with all def-std-pkg.sh packages and there was no problem when doing ./PklFonts > FontData.pkl However the Juno executable does not run, but this error shows: admin07 at sl07:~$ Juno *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "../src/runtime/common/RTCollector.m3", line 361 *** Cancelado admin07 at sl07:~$ Thanks --- "Daniel Alejandro Benavides D." wrote: > > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > ______________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From hosking at cs.purdue.edu Tue Apr 10 02:15:59 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 9 Apr 2007 20:15:59 -0400 Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> References: <20070409234925.11036.qmail@web27106.mail.ukl.yahoo.com> Message-ID: <9ED98160-9379-4A83-951B-F07959ECDBEB@cs.purdue.edu> This looks like something wrong with your build. Please run "do-cm3- std.sh realclean" before "buildship". On Apr 9, 2007, at 7:49 PM, Daniel Alejandro Benavides D. wrote: > Hello: > I have updated the libfreetype6 (to libfreetype6 > 2.2.1-5) library in ubuntu 6.10, and its now corrected > the problem on juno with PklFonts, because, I have > executed do-cm3-std.sh buildship with all > def-std-pkg.sh packages and there was no problem when > doing ./PklFonts > FontData.pkl > > However the Juno executable does not run, but this > error shows: > > admin07 at sl07:~$ Juno > > > *** > *** runtime error: > *** An enumeration or subrange value was out of > range. > *** file "../src/runtime/common/RTCollector.m3", > line 361 > *** > > Cancelado > admin07 at sl07:~$ > > > Thanks > > --- "Daniel Alejandro Benavides D." > wrote: > >> >> >> I have exectuted do-cm3-std.sh buildship with the >> current cvs sources with cm3-5.4.0 bootstrap and in >> just one package I get a problem; Juno-2 with >> ubuntu >> 6.10, when tries to do a redirection of the of the >> PklFonts executable. >> ./PklFonts > FontData.pkl >> >> It got segmentation fault >> >> The thing is, after the rest of the system is >> compiled >> the Juno-2 compiles with no problems, including that >> redirection. >> >> > > > > ______________________________________________ > LLama Gratis a cualquier PC del Mundo. > Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. > http://es.voice.yahoo.com > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Tue Apr 10 04:30:46 2007 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 10 Apr 2007 04:30:46 +0200 (CEST) Subject: [M3devel] Re: installation on LINUXLIBC6: In-Reply-To: <461AED47.8090906@elego.de> Message-ID: <20070410023046.3362.qmail@web27103.mail.ukl.yahoo.com> Hi: Can you check the version of libfreetype6? Mus be 2.2.1-5. The other you can do to check Juno-2 is cd to the path of juno and build it manually with cm3 -ship, in the right order of the directories: pkl-fonts juno-machine juno-compiler juno-app > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. Mentor does not depend on juno-2. What is the problem compiling mentor? Thanks --- Neels Janosch Hofmeyr wrote: > having gotten this useful information: > > I have exectuted do-cm3-std.sh buildship with the > current cvs sources with cm3-5.4.0 bootstrap and in > just one package I get a problem; Juno-2 with > ubuntu > 6.10, when tries to do a redirection of the of the > PklFonts executable. > ./PklFonts > FontData.pkl > > It got segmentation fault > > The thing is, after the rest of the system is > compiled > the Juno-2 compiles with no problems, including that > redirection. > > > I pondered a bit to find the right way of un-listing > juno-2. In the end > it turned out that I can comment lines 149 to 152 > from > scripts/def-std-pkgs.sh: > > # The Juno-2 graphical constraint based editor > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} pkl-fonts" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-machine" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-compiler" > #[ "${M3OSTYPE}" != "WIN32" -o -n "${CM3_ALL}" ] && > P="${P} juno-app" > > and then I found I also needed to comment out line > 158: > #P="${P} mentor" > > in order for cm3-std to compile and install. > > However, undoing the comments and compiling again > did not succeed. I > guess I'll have to live without the Juno-2 graphical > constraint based > editor... unless you have another suggestion, that > is. > > thanks, > neels > > Neels Janosch Hofmeyr wrote: > > Sorry, no success here... Do I really need this > juno thing? > > > > [...] > > === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > > +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > > -DROOT='/home/neels/elego/tmp/cm3' +++ > > --- building in LINUXLIBC6 --- > > > > ignoring ../src/m3overrides > > > > cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > > Segmentation fault (core dumped) > > > "/home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile", > line > > 24: quake runtime error: exit 139: cd > ../pkl-fonts/LINUXLIBC6 && > > ./PklFonts > FontData.pkl > > > > --procedure-- -line- -file--- > > exec -- > > include_dir 24 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/src/m3makefile > > 5 > > > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app/LINUXLIBC6/m3make.args > > > > Fatal Error: package build failed > > *** execution of failed *** > > > > $ env | grep LD > > LD_POINTER_GUARD=0 > > > > > > Stefan Sperling wrote: > > > >> On Mon, Apr 09, 2007 at 02:38:34PM +0200, Neels > Janosch Hofmeyr wrote: > >> > >> > >>> More problems have shown up with cm3 5.4.0 on my > ubuntu 6.10: > >>> > >>> " > >>> === package > /home/neels/elego/tmp/cm3/m3-ui/juno-2/juno-app === > >>> +++ cm3 -build > -DROOT='/home/neels/elego/tmp/cm3' && cm3 -ship > >>> -DROOT='/home/neels/elego/tmp/cm3' +++ > >>> --- building in LINUXLIBC6 --- > >>> > >>> ignoring ../src/m3overrides > >>> > >>> cd ../pkl-fonts/LINUXLIBC6 && ./PklFonts > > FontData.pkl > >>> Segmentation fault (core dumped) > >>> > >>> > >> Try this (from known problems page): > >> > >> Platforms: Fedora Core Linux, maybe other > distributions > >> > >> Description: All Modula3 programs (including cm3 > itself) exit with a > >> segmentation violation. > >> > >> Solution: Set the LD_POINTER_GUARD environment > variable to 0 before > >> running Modula3 programs, using a command like > "export > >> LD_POINTER_GUARD=0" or equivalent. > >> > >> > >> > > > > > > -- > Neels Janosch Hofmeyr > Software Developer > > neels at elego.de > Public Key: > http://binarchy.net/neels/neels.hofmeyr.public.key.asc > > elego Software Solutions GmbH > Ohmstra?e 9, 10179 Berlin HRB 77719 > Tel.: +49 30 23 45 86 96 Amtsgericht > Charlottenburg > Fax: +49 30 23 45 86 95 Sitz der > Gesellschaft: Berlin > http://www.elegosoft.com > Gesch?ftsf?hrer: Olaf Wagner > > > ____________________________________________________________________________________ LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y m?viles desde 1 c?ntimo por minuto. http://es.voice.yahoo.com From darko at darko.org Tue Apr 10 21:01:54 2007 From: darko at darko.org (Darko) Date: Tue, 10 Apr 2007 21:01:54 +0200 Subject: [M3devel] Native M3 Implementation on Symbian Phones Message-ID: <07E52AF0-4E02-4AB5-88AE-41E4C1BC7F75@darko.org> It's interesting to note that Nokia have released 'OpenC' libraries < http://forum.nokia.com/openc > which seem to provide Posix like runtime and library support. There are limitations which may cause problems with the standard M3 runtime, but its certainly a step closer to being able to target such devices. Darko. From darko at darko.org Wed Apr 18 03:51:27 2007 From: darko at darko.org (Darko) Date: Wed, 18 Apr 2007 03:51:27 +0200 Subject: [M3devel] RTHooks CheckStoreTraced and CheckLoadTraced Message-ID: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Hi, Wondering if you can explain the use of these calls a little more. I'm currently using type maps to read and write fields from traced objects. Reading a traced reference from inside a traced object into a local variable is not working as it should. Should I use CheckLoadTraced and if so when and how? Looking at your changes to RTTypeMap, writing references into objects means you need to call CheckStoreTraced on the object written inside of, before it is written? Cheers, Darko. From wagner at plane.elego.de Wed Apr 18 08:04:18 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Wed, 18 Apr 2007 08:04:18 +0200 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] Message-ID: <20070418060417.GB14308@elegosoft.com> Is this the stack encryption problem again? Has anybody tried on Suse 10.1? Olaf ----- Forwarded message from Pino Zollo ----- DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= From: Pino Zollo To: m3-support at elego.de Subject: Installation Problem Date: Tue, 17 Apr 2007 18:00:50 -0400 X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no version=3.0.3 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de X-Virus-Status: Clean Dear Sirs, I have installed the last version of cm3 but I have the following problem: cm3 compiles all programs, but are correctly executed only the ones compiled with the option build_standalone(). The programs which use shared libraries end in segmentation fault. Also the script do-cm3-std.sh fails to compile everything for the same problem: the program PklFonts in the directory m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault because needs shared libraries. Any suggestion ? I am using the distribution SuSE 10.1 with kernel 2.4.27 In the installation I have found that the command "export" was not doing properly its job, so I had to put the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the path in /etc/ld.so.conf I noticed that executables contain the string "/lib/ld-linux.so.2" so I had to make a symbolic link in /lib as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. I have also tried export "LD_POINTER_GUARD=0" also without any change. I have done quite a lot on Modula-3 programming years ago when kernel was 1.2.13. I would like to make public my old sources under the GNU licence. For this reason I am willing to check them on the new version on m3. I am available for doing development in Modula-3 in the limits of my capabilities. My name is Pino Zollo and my e-mail is pinozollo at gmail.com Some of my works are on http://www.qsl.net/zp4kfx and a description of my profesional experience is on http://www.qsl.net/zp4kfx/CV Thanks for the attencion. Best regards Pino ----- End forwarded message ----- -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 18 15:25:04 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 09:25:04 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > It seems I need a check on load of a traced reference in RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once I've checked it in (the CVS mailer is broken again I think). > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:34:58 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:34:58 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: I assume you have an apply method for your RTTypeMap.Visitor that takes "field: ADDRESS" and treats it as "REF REFANY". This is wrong. When reading a REF field you should use the following idiom: WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO ... access field via ref^ ... END; This will automatically insert a call to the appropriate runtime routines on accessing the reference field. There should be no need for you to call the runtime routines directly. On Apr 17, 2007, at 9:51 PM, Darko wrote: > Hi, > > Wondering if you can explain the use of these calls a little more. > I'm currently using type maps to read and write fields from traced > objects. Reading a traced reference from inside a traced object > into a local variable is not working as it should. Should I use > CheckLoadTraced and if so when and how? Looking at your changes to > RTTypeMap, writing references into objects means you need to call > CheckStoreTraced on the object written inside of, before it is > written? > > Cheers, > Darko. From hosking at cs.purdue.edu Wed Apr 18 16:40:07 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:40:07 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: PS You should not use "REF REFANY", since a "REF" is assumed by the runtime system to be a *tidy* pointer to an object in the heap. Clearly, the address of a field inside an object is by definition *untidy*. On Apr 18, 2007, at 10:34 AM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:43:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:43:32 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <5E2DE156-C048-4E0B-A471-2A096BABA306@cs.purdue.edu> PS There is no fix to RTTypeMap needed here so long as you use the correct idiom as described earlier. This is unsafe code so you are expected to behave properly by the runtime system. On Apr 18, 2007, at 9:25 AM, Tony Hosking wrote: > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> > > It seems I need a check on load of a traced reference in > RTTypeMap.Walk. SHould be a straightforward fix. I'll e-mail once > I've checked it in (the CVS mailer is broken again I think). > >> Cheers, >> Darko. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Wed Apr 18 16:50:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 10:50:08 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <5AB076A1-4E10-4BD0-8450-12E7B0F9B7B8@cs.purdue.edu> Does kernel 2.4 support NPTL? On Apr 18, 2007, at 2:04 AM, Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message- > id:from:to:subject:date:mime-version:content-type:content-transfer- > encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/ > WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQM > JJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ > U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 > tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on > plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on > plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following > problem: > > cm3 compiles all programs, but are correctly executed only the ones > compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not > doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did > put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" > so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any > change. > > I have done quite a lot on Modula-3 programming years ago when > kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to > check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a > description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney.bates at wichita.edu Wed Apr 18 18:15:58 2007 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Wed, 18 Apr 2007 11:15:58 -0500 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <20070418060417.GB14308@elegosoft.com> References: <20070418060417.GB14308@elegosoft.com> Message-ID: <4626443E.8040704@wichita.edu> This symptom (works when statically linked, segfaults during startup when using dynamic libraries) is, as I remember, the symptom of the problem with changes in newer setjmp/longjmp that I posted about on January 11. The newer setjmp/longjmp mangle and demangle a couple of the registers (apparently to prevent some security exploit). When used as a pair, the change is harmless. But some uses of setjmp in the M3 libraries actually use the contents of the setjmp buffer to get register contents for thread switching, etc. There is an assembly-coded special version of setjmp in the M3 code, dating from many years back, that works with older longjmp, but not with the newest. Tony committed a change that removes this version of setjmp. but I don't think that fixes all the problems in all cases. I think existing uses of setjmp have to be split into two categories: those that just use the result to pass back to longjmp, and those that use the result for fetching registers. Then different variants of setjmp have to be used in the two cases. Antony suggested getcontext/setcontext, but I guess there are portability problems here? I looked at this briefly, and there are a very large number of uses of setjmp to check out. I have thought about working on it, but have been busy with other things for a while. I never got -DPTHREAD working and ran out of time on that approach. As I recall, -DPTHREAD is now the default, so I would guess Pino's installation is using it. Olaf Wagner wrote: > Is this the stack encryption problem again? Has anybody tried on > Suse 10.1? > > Olaf > > ----- Forwarded message from Pino Zollo ----- > > DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; > d=gmail.com; s=beta; > h=domainkey-signature:received:received:message-id:from:to:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; > b=U78B+w8R6V2/WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQMJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/U4PrfCQw+NQLqPP2k0Vwncv7Ss= > From: Pino Zollo > To: m3-support at elego.de > Subject: Installation Problem > Date: Tue, 17 Apr 2007 18:00:50 -0400 > X-Spam-Status: No, score=2.7 required=5.0 tests=DEAR_SOMETHING,RCVD_BY_IP, > RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no > version=3.0.3 > X-Spam-Level: ** > X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on plane.elego.de > X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on plane.elego.de > X-Virus-Status: Clean > > Dear Sirs, > I have installed the last version of cm3 but I have the following problem: > > cm3 compiles all programs, but are correctly executed only the ones compiled > with the option build_standalone(). > The programs which use shared libraries end in segmentation fault. > Also the script do-cm3-std.sh fails to compile everything for the same > problem: the program PklFonts in the directory > m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault > because needs shared libraries. > > Any suggestion ? > > I am using the distribution SuSE 10.1 with kernel 2.4.27 > > In the installation I have found that the command "export" was not doing > properly its job, so I had to put > the path /usr/local/cm3/bin in /etc/profile. For libraries I did put the > path in /etc/ld.so.conf > > I noticed that executables contain the string "/lib/ld-linux.so.2" so I had > to make a symbolic link in /lib > as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. > > I have also tried export "LD_POINTER_GUARD=0" also without any change. > > I have done quite a lot on Modula-3 programming years ago when kernel was > 1.2.13. I would like to make public my > old sources under the GNU licence. For this reason I am willing to check > them on the new version on m3. > > I am available for doing development in Modula-3 in the limits of my > capabilities. > > My name is Pino Zollo and my e-mail is pinozollo at gmail.com > Some of my works are on http://www.qsl.net/zp4kfx and a description of my > profesional experience is on > http://www.qsl.net/zp4kfx/CV > > Thanks for the attencion. > Best regards > > Pino > > > ----- End forwarded message ----- > -- ------------------------------------------------------------- Rodney M. Bates, retired assistant professor Dept. of Computer Science, Wichita State University Wichita, KS 67260-0083 316-978-3922 rodney.bates at wichita.edu From hosking at cs.purdue.edu Wed Apr 18 18:33:06 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 12:33:06 -0400 Subject: [M3devel] [pinozollo@gmail.com: Installation Problem] In-Reply-To: <4626443E.8040704@wichita.edu> References: <20070418060417.GB14308@elegosoft.com> <4626443E.8040704@wichita.edu> Message-ID: Yes, use of setjmp/longjmp for *user*-level threads is bound to fail in the presence of the security mangling they now do, since new thread contexts are obtained by cloning a setjmp buffer from the root thread, and then hacking its stack pointer appropriately to match the new thread. This doesn't work with the secure setjmp/longjmp implementations. On platforms like Linux where setjmp/longjmp mangling prevent their use for user-level threading the only alternative is to use the explicit makecontext with getcontext/ setcontext. This is how *user*-level threading is done on Solaris. With the newer PTHREAD threading (for SOLgnu, SOLsun, LINUXLIBC6, I386_DARWIN, PPC_DARWIN) this should not be an issue, but it does require NPTL on Linux platforms (I don't think 2.4 kernels support NPTL). I do know that I can successfully build from the current CVS tree on recent releases of Fedora Core, without problems. On Apr 18, 2007, at 12:15 PM, Rodney M. Bates wrote: > This symptom (works when statically linked, segfaults during > startup when > using dynamic libraries) is, as I remember, the symptom of the > problem with > changes in newer setjmp/longjmp that I posted about on January 11. > > The newer setjmp/longjmp mangle and demangle a couple of the registers > (apparently to prevent some security exploit). When used as a > pair, the > change is harmless. But some uses of setjmp in the M3 libraries > actually > use the contents of the setjmp buffer to get register contents for > thread > switching, etc. > > There is an assembly-coded special version of setjmp in the M3 > code, dating > from many years back, that works with older longjmp, but not with > the newest. > Tony committed a change that removes this version of setjmp. but I > don't think > that fixes all the problems in all cases. I don't see problems with this change. What problems are you seeing? In this case, setjmp/longjmp is only being used for exception handling, assuming threads are using PTHREAD (i.e., Linux NPTL). > > I think existing uses of setjmp have to be split into two > categories: those > that just use the result to pass back to longjmp, and those that > use the result > for fetching registers. Then different variants of setjmp have to > be used in > the two cases. Antony suggested getcontext/setcontext, but I guess > there are > portability problems here? > > I looked at this briefly, and there are a very large number of uses > of setjmp > to check out. I have thought about working on it, but have been > busy with > other things for a while. > > I never got -DPTHREAD working and ran out of time on that approach. > As I recall, > -DPTHREAD is now the default, so I would guess Pino's installation > is using it. > > Olaf Wagner wrote: >> Is this the stack encryption problem again? Has anybody tried on >> Suse 10.1? >> Olaf >> ----- Forwarded message from Pino Zollo ----- >> DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; >> d=gmail.com; s=beta; >> h=domainkey-signature:received:received:message- >> id:from:to:subject:date:mime-version:content-type:content-transfer- >> encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; >> b=U78B+w8R6V2/ >> WUP4jo0Lv4HJ9iGXHjfGLELvNGPtGjJK9Opw1mlxgXFRzGFuvesKInntAGKb4U0ZfiGWQ >> MJJ/wEFq0CsNyAhxU6OGNf60yMXqTx825qudBwKtMm/9IqA8K1RCKBYjk0GGVsh/ >> U4PrfCQw+NQLqPP2k0Vwncv7Ss= >> From: Pino Zollo >> To: m3-support at elego.de >> Subject: Installation Problem >> Date: Tue, 17 Apr 2007 18:00:50 -0400 >> X-Spam-Status: No, score=2.7 required=5.0 >> tests=DEAR_SOMETHING,RCVD_BY_IP, >> RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_PASS,SPF_PASS autolearn=no >> version=3.0.3 >> X-Spam-Level: ** >> X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on >> plane.elego.de >> X-Virus-Scanned: ClamAV 0.88.7/3109/Tue Apr 17 05:59:17 2007 on >> plane.elego.de >> X-Virus-Status: Clean >> Dear Sirs, >> I have installed the last version of cm3 but I have the following >> problem: >> cm3 compiles all programs, but are correctly executed only the >> ones compiled >> with the option build_standalone(). >> The programs which use shared libraries end in segmentation fault. >> Also the script do-cm3-std.sh fails to compile everything for the >> same >> problem: the program PklFonts in the directory >> m3-ui/juno-2/juno-app/pkl-fonts/LINUXLIBC6 ends in segmentation fault >> because needs shared libraries. >> Any suggestion ? >> I am using the distribution SuSE 10.1 with kernel 2.4.27 >> In the installation I have found that the command "export" was not >> doing >> properly its job, so I had to put >> the path /usr/local/cm3/bin in /etc/profile. For libraries I did >> put the >> path in /etc/ld.so.conf >> I noticed that executables contain the string "/lib/ld-linux.so.2" >> so I had >> to make a symbolic link in /lib >> as "ln -s ld-2.4.so ld-linux.so.2" but nothing did change. >> I have also tried export "LD_POINTER_GUARD=0" also without any >> change. >> I have done quite a lot on Modula-3 programming years ago when >> kernel was >> 1.2.13. I would like to make public my >> old sources under the GNU licence. For this reason I am willing to >> check >> them on the new version on m3. >> I am available for doing development in Modula-3 in the limits of my >> capabilities. >> My name is Pino Zollo and my e-mail is pinozollo at gmail.com >> Some of my works are on http://www.qsl.net/zp4kfx and a >> description of my >> profesional experience is on >> http://www.qsl.net/zp4kfx/CV >> Thanks for the attencion. >> Best regards >> Pino >> ----- End forwarded message ----- > > -- > ------------------------------------------------------------- > Rodney M. Bates, retired assistant professor > Dept. of Computer Science, Wichita State University > Wichita, KS 67260-0083 > 316-978-3922 > rodney.bates at wichita.edu > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From darko at darko.org Thu Apr 19 03:37:10 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 03:37:10 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> Message-ID: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Not actually using that code, but I did see your helpful commit on it some time ago. It seems that the problem I described earlier is due to another bug. The code I'm working on does a lot of reading and writing of traced references and I was hoping to get a better understanding of situations I need to be aware of and how to use those RTHooks calls more efficiently. So say I have two traced objects and I am copying a traced reference from a field in one to the other. What set of calls would be normally required there? I'm sure you haven't got time to go into the minutiae of the runtime system, but any hints would be appreciated. Cheers, Darko. On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > I assume you have an apply method for your RTTypeMap.Visitor that > takes "field: ADDRESS" and treats it as "REF REFANY". This is > wrong. When reading a REF field you should use the following idiom: > > WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO > ... access field via ref^ ... > END; > > This will automatically insert a call to the appropriate runtime > routines on accessing the reference field. > > There should be no need for you to call the runtime routines directly. > > On Apr 17, 2007, at 9:51 PM, Darko wrote: > >> Hi, >> >> Wondering if you can explain the use of these calls a little more. >> I'm currently using type maps to read and write fields from traced >> objects. Reading a traced reference from inside a traced object >> into a local variable is not working as it should. Should I use >> CheckLoadTraced and if so when and how? Looking at your changes to >> RTTypeMap, writing references into objects means you need to call >> CheckStoreTraced on the object written inside of, before it is >> written? >> >> Cheers, >> Darko. > From hosking at cs.purdue.edu Thu Apr 19 04:05:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:05:42 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. You SHOULD NOT need to use these calls so long as you are accessing using properly typed references (as in my example earlier). This is not an issue for SAFE code, only in UNSAFE modules must you be careful how you use LOOPHOLE to cast references. > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? How are you copying the references? I would like to see your code for this. So long as you properly type things then the calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be generated by the compiler. Anyway... RTHooks.CheckLoadTracedRef is generated by the compiler whenever you access a traced reference in memory (e.g., via dereference or from a shared variable). Its job is to prevent mutators from acquiring references to "gray" objects (that are reachable but still to be scanned for the traced references that they contain). The check turns the object from gray to black by scanning its references and making sure none of them refer to white objects (that are not known to be reachable by the collector). RTHooks.CheckStoreTraced is generated by the compiler whenever a traced reference is stored (or might be stored, e.g., for VAR) into a traced object. The check makes sure that the generational GC knows that the object has been modified. It needs to know this for objects in the older genration. Again, I reiterate that it is not my intention that programmers actually call these runtime hooks explicitly! I am certain I can rewrite any code you might have so that you do not need to call them explicitly. Instead, properly typed references will result in the checks being generated for you by the compiler. Please feel free to send me code snippets for review. > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From hosking at cs.purdue.edu Thu Apr 19 04:09:35 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 18 Apr 2007 22:09:35 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> Message-ID: <55BF1964-E58E-4E20-93F7-9DDA1A7B1CF4@cs.purdue.edu> On Apr 18, 2007, at 9:37 PM, Darko wrote: > Not actually using that code, but I did see your helpful commit on > it some time ago. It seems that the problem I described earlier is > due to another bug. The code I'm working on does a lot of reading > and writing of traced references and I was hoping to get a better > understanding of situations I need to be aware of and how to use > those RTHooks calls more efficiently. > > So say I have two traced objects and I am copying a traced > reference from a field in one to the other. What set of calls would > be normally required there? If you have: VAR source, target: OBJECT field: REFANY END; then copying field from source to target is written as normal: target.field := source.field; The compiler will emit the appropriate calls: CheckLoadTracedRef(source.field) CheckStoreTraced(target) Why do you need to call these runtime routines explicitly if the compiler will generate them for you? > > I'm sure you haven't got time to go into the minutiae of the > runtime system, but any hints would be appreciated. > > Cheers, > Darko. > > > On 18/04/2007, at 4:34 PM, Tony Hosking wrote: > >> I assume you have an apply method for your RTTypeMap.Visitor that >> takes "field: ADDRESS" and treats it as "REF REFANY". This is >> wrong. When reading a REF field you should use the following idiom: >> >> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >> ... access field via ref^ ... >> END; >> >> This will automatically insert a call to the appropriate runtime >> routines on accessing the reference field. >> >> There should be no need for you to call the runtime routines >> directly. >> >> On Apr 17, 2007, at 9:51 PM, Darko wrote: >> >>> Hi, >>> >>> Wondering if you can explain the use of these calls a little >>> more. I'm currently using type maps to read and write fields from >>> traced objects. Reading a traced reference from inside a traced >>> object into a local variable is not working as it should. Should >>> I use CheckLoadTraced and if so when and how? Looking at your >>> changes to RTTypeMap, writing references into objects means you >>> need to call CheckStoreTraced on the object written inside of, >>> before it is written? >>> >>> Cheers, >>> Darko. >> From darko at darko.org Thu Apr 19 04:49:29 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 04:49:29 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> Message-ID: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Thanks, that's very helpful. I agree I shouldn't need it but it's exactly because I'm not using the compiler is why I need to know about it. The code is attached, it's still in development and unfinished but you'll get the idea. Basically the module allows you to programatically access structure fields. It allows for applications like being able to create indexes (using an adaptation of the Table code) of objects using arbitrary fields as keys. It makes dynamic programming a bit more dynamic. The code is intended to be completely safe, but there's more checks I need to do. -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.m3 Type: application/octet-stream Size: 41052 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: InType.i3 Type: application/octet-stream Size: 6117 bytes Desc: not available URL: -------------- next part -------------- On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > > On Apr 18, 2007, at 9:37 PM, Darko wrote: > >> Not actually using that code, but I did see your helpful commit on >> it some time ago. It seems that the problem I described earlier is >> due to another bug. The code I'm working on does a lot of reading >> and writing of traced references and I was hoping to get a better >> understanding of situations I need to be aware of and how to use >> those RTHooks calls more efficiently. > > You SHOULD NOT need to use these calls so long as you are accessing > using properly typed references (as in my example earlier). This > is not an issue for SAFE code, only in UNSAFE modules must you be > careful how you use LOOPHOLE to cast references. > >> So say I have two traced objects and I am copying a traced >> reference from a field in one to the other. What set of calls >> would be normally required there? > > How are you copying the references? I would like to see your code > for this. So long as you properly type things then the calls to > RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be > generated by the compiler. > > Anyway... > > RTHooks.CheckLoadTracedRef is generated by the compiler whenever > you access a traced reference in memory (e.g., via dereference or > from a shared variable). Its job is to prevent mutators from > acquiring references to "gray" objects (that are reachable but > still to be scanned for the traced references that they contain). > The check turns the object from gray to black by scanning its > references and making sure none of them refer to white objects > (that are not known to be reachable by the collector). > > RTHooks.CheckStoreTraced is generated by the compiler whenever a > traced reference is stored (or might be stored, e.g., for VAR) into > a traced object. The check makes sure that the generational GC > knows that the object has been modified. It needs to know this for > objects in the older genration. > > Again, I reiterate that it is not my intention that programmers > actually call these runtime hooks explicitly! I am certain I can > rewrite any code you might have so that you do not need to call > them explicitly. Instead, properly typed references will result in > the checks being generated for you by the compiler. Please feel > free to send me code snippets for review. > >> I'm sure you haven't got time to go into the minutiae of the >> runtime system, but any hints would be appreciated. >> >> Cheers, >> Darko. >> >> >> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >> >>> I assume you have an apply method for your RTTypeMap.Visitor that >>> takes "field: ADDRESS" and treats it as "REF REFANY". This is >>> wrong. When reading a REF field you should use the following idiom: >>> >>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>> ... access field via ref^ ... >>> END; >>> >>> This will automatically insert a call to the appropriate runtime >>> routines on accessing the reference field. >>> >>> There should be no need for you to call the runtime routines >>> directly. >>> >>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>> >>>> Hi, >>>> >>>> Wondering if you can explain the use of these calls a little >>>> more. I'm currently using type maps to read and write fields >>>> from traced objects. Reading a traced reference from inside a >>>> traced object into a local variable is not working as it should. >>>> Should I use CheckLoadTraced and if so when and how? Looking at >>>> your changes to RTTypeMap, writing references into objects means >>>> you need to call CheckStoreTraced on the object written inside >>>> of, before it is written? >>>> >>>> Cheers, >>>> Darko. >>> > From hosking at cs.purdue.edu Thu Apr 19 15:34:40 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 09:34:40 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> Message-ID: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> My immediate reaction is that you probably want to approach reads slightly differently. Instead of using your generic InTypeRead routine to access the appropriate bits/bytes of the value, you probably want to replace it with some sort of simple addressing mechanism that computes the raw address of the field. Traced reference fields must always be aligned within the heap object they are stored in so your addressing for these will not need bit offsets. Once you have an ADDRESS "addr" for the traced reference field, so long as you use the idiom I gave earlier, you will get the correct behavior for reads. That is, WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO .. use field^ to access the traced reference stored at address field .. END; The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the traced reference "r" once you have loopholed it through the raw field access into a REFANY, before you return it from InTypeReadRef: PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): REFANY = VAR v: REFANY; BEGIN this.check(field, Kind.Ref, obj); this.read(obj, field, ADR(v), BYTESIZE(v)); RTHooks.CheckLoadTracedRef(v); RETURN v; END InTypeReadRef; This is safe for the current CM3 ambiguous roots collector, but this code would probably break with a fully copying/relocating collector (if one was ever implemented for Modula-3), since the raw bytes of the reference might become stale between the point you load them from memory and the point they are placed in the typed REFANY variable "v" (effectively, you'd be hiding a reference from the GC for a brief period, and it would not be able to update that hidden reference in the face of its target being moved). I note, generally, that your field addressing mechanisms only work anyway because we are using an ambiguous roots collector that pins all objects referenced from the thread stacks (your "obj" parameters in the accessor methods), which allows the use of RTHeap.GetDataAdr to provide an address for the object that is valid while the object reference is held on the stack. When writing references to object fields you will need to invoke "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is being stored. Thus, for InTypeWriteRef: PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: CARDINAL) = BEGIN this.check(field, Kind.Ref, obj); WITH f = this.offs.get(field) DO IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; END; this.write(obj, field, ADR(val), BYTESIZE(val)); RTHooks.CheckStoreTraced(obj); END InTypeWriteRef; If you are performing multiple writes of reference fields on the same object (e.g., to an array) then you can invoke CheckStoreTraced once for the whole object. Hope this helps. On Apr 18, 2007, at 10:49 PM, Darko wrote: > Thanks, that's very helpful. I agree I shouldn't need it but it's > exactly because I'm not using the compiler is why I need to know > about it. The code is attached, it's still in development and > unfinished but you'll get the idea. > > Basically the module allows you to programatically access structure > fields. It allows for applications like being able to create > indexes (using an adaptation of the Table code) of objects using > arbitrary fields as keys. It makes dynamic programming a bit more > dynamic. The code is intended to be completely safe, but there's > more checks I need to do. > > > > > > > > > > On 19/04/2007, at 4:05 AM, Tony Hosking wrote: > >> >> On Apr 18, 2007, at 9:37 PM, Darko wrote: >> >>> Not actually using that code, but I did see your helpful commit >>> on it some time ago. It seems that the problem I described >>> earlier is due to another bug. The code I'm working on does a lot >>> of reading and writing of traced references and I was hoping to >>> get a better understanding of situations I need to be aware of >>> and how to use those RTHooks calls more efficiently. >> >> You SHOULD NOT need to use these calls so long as you are >> accessing using properly typed references (as in my example >> earlier). This is not an issue for SAFE code, only in UNSAFE >> modules must you be careful how you use LOOPHOLE to cast references. >> >>> So say I have two traced objects and I am copying a traced >>> reference from a field in one to the other. What set of calls >>> would be normally required there? >> >> How are you copying the references? I would like to see your code >> for this. So long as you properly type things then the calls to >> RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be >> generated by the compiler. >> >> Anyway... >> >> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >> you access a traced reference in memory (e.g., via dereference or >> from a shared variable). Its job is to prevent mutators from >> acquiring references to "gray" objects (that are reachable but >> still to be scanned for the traced references that they contain). >> The check turns the object from gray to black by scanning its >> references and making sure none of them refer to white objects >> (that are not known to be reachable by the collector). >> >> RTHooks.CheckStoreTraced is generated by the compiler whenever a >> traced reference is stored (or might be stored, e.g., for VAR) >> into a traced object. The check makes sure that the generational >> GC knows that the object has been modified. It needs to know this >> for objects in the older genration. >> >> Again, I reiterate that it is not my intention that programmers >> actually call these runtime hooks explicitly! I am certain I can >> rewrite any code you might have so that you do not need to call >> them explicitly. Instead, properly typed references will result >> in the checks being generated for you by the compiler. Please >> feel free to send me code snippets for review. >> >>> I'm sure you haven't got time to go into the minutiae of the >>> runtime system, but any hints would be appreciated. >>> >>> Cheers, >>> Darko. >>> >>> >>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>> >>>> I assume you have an apply method for your RTTypeMap.Visitor >>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>> This is wrong. When reading a REF field you should use the >>>> following idiom: >>>> >>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>> ... access field via ref^ ... >>>> END; >>>> >>>> This will automatically insert a call to the appropriate runtime >>>> routines on accessing the reference field. >>>> >>>> There should be no need for you to call the runtime routines >>>> directly. >>>> >>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>> >>>>> Hi, >>>>> >>>>> Wondering if you can explain the use of these calls a little >>>>> more. I'm currently using type maps to read and write fields >>>>> from traced objects. Reading a traced reference from inside a >>>>> traced object into a local variable is not working as it >>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>> Looking at your changes to RTTypeMap, writing references into >>>>> objects means you need to call CheckStoreTraced on the object >>>>> written inside of, before it is written? >>>>> >>>>> Cheers, >>>>> Darko. >>>> >> > From darko at darko.org Thu Apr 19 19:09:45 2007 From: darko at darko.org (Darko) Date: Thu, 19 Apr 2007 19:09:45 +0200 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: Thanks, thats very useful. Garbage collection is a bit of a mystery, since I can't imagine myself changing it I've never studied it, but it's on my mind constantly. M3 is so safe whenever something weird happens with my code I know it's because I've trodden on a traced reference, or copied it into a untraced space and back or something. I've relied on the pinning of references that appear in the stack extensively, and I think a lot of other code does too. It a bit of a worry to think that it's possible it might disappear. If we don't have this, short of stopping the collector, what do you do if you want your object to stay still? On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > My immediate reaction is that you probably want to approach reads > slightly differently. Instead of using your generic InTypeRead > routine to access the appropriate bits/bytes of the value, you > probably want to replace it with some sort of simple addressing > mechanism that computes the raw address of the field. Traced > reference fields must always be aligned within the heap object they > are stored in so your addressing for these will not need bit > offsets. Once you have an ADDRESS "addr" for the traced reference > field, so long as you use the idiom I gave earlier, you will get > the correct behavior for reads. That is, > > WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO > .. use field^ to access the traced reference stored at address > field .. > END; > > The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the > traced reference "r" once you have loopholed it through the raw > field access into a REFANY, before you return it from InTypeReadRef: > > PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): > REFANY = > VAR v: REFANY; > BEGIN > this.check(field, Kind.Ref, obj); > this.read(obj, field, ADR(v), BYTESIZE(v)); > RTHooks.CheckLoadTracedRef(v); > RETURN v; > END InTypeReadRef; > > This is safe for the current CM3 ambiguous roots collector, but > this code would probably break with a fully copying/relocating > collector (if one was ever implemented for Modula-3), since the raw > bytes of the reference might become stale between the point you > load them from memory and the point they are placed in the typed > REFANY variable "v" (effectively, you'd be hiding a reference from > the GC for a brief period, and it would not be able to update that > hidden reference in the face of its target being moved). > > I note, generally, that your field addressing mechanisms only work > anyway because we are using an ambiguous roots collector that pins > all objects referenced from the thread stacks (your "obj" > parameters in the accessor methods), which allows the use of > RTHeap.GetDataAdr to provide an address for the object that is > valid while the object reference is held on the stack. > > When writing references to object fields you will need to invoke > "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is > being stored. Thus, for InTypeWriteRef: > > PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: > CARDINAL) = > BEGIN > this.check(field, Kind.Ref, obj); > WITH f = this.offs.get(field) DO > IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE > (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; > END; > this.write(obj, field, ADR(val), BYTESIZE(val)); > RTHooks.CheckStoreTraced(obj); > END InTypeWriteRef; > > If you are performing multiple writes of reference fields on the > same object (e.g., to an array) then you can invoke > CheckStoreTraced once for the whole object. > > Hope this helps. > > On Apr 18, 2007, at 10:49 PM, Darko wrote: > >> Thanks, that's very helpful. I agree I shouldn't need it but it's >> exactly because I'm not using the compiler is why I need to know >> about it. The code is attached, it's still in development and >> unfinished but you'll get the idea. >> >> Basically the module allows you to programatically access >> structure fields. It allows for applications like being able to >> create indexes (using an adaptation of the Table code) of objects >> using arbitrary fields as keys. It makes dynamic programming a bit >> more dynamic. The code is intended to be completely safe, but >> there's more checks I need to do. >> >> >> >> >> >> >> >> >> >> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >> >>> >>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>> >>>> Not actually using that code, but I did see your helpful commit >>>> on it some time ago. It seems that the problem I described >>>> earlier is due to another bug. The code I'm working on does a >>>> lot of reading and writing of traced references and I was hoping >>>> to get a better understanding of situations I need to be aware >>>> of and how to use those RTHooks calls more efficiently. >>> >>> You SHOULD NOT need to use these calls so long as you are >>> accessing using properly typed references (as in my example >>> earlier). This is not an issue for SAFE code, only in UNSAFE >>> modules must you be careful how you use LOOPHOLE to cast references. >>> >>>> So say I have two traced objects and I am copying a traced >>>> reference from a field in one to the other. What set of calls >>>> would be normally required there? >>> >>> How are you copying the references? I would like to see your >>> code for this. So long as you properly type things then the >>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>> will be generated by the compiler. >>> >>> Anyway... >>> >>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>> you access a traced reference in memory (e.g., via dereference or >>> from a shared variable). Its job is to prevent mutators from >>> acquiring references to "gray" objects (that are reachable but >>> still to be scanned for the traced references that they >>> contain). The check turns the object from gray to black by >>> scanning its references and making sure none of them refer to >>> white objects (that are not known to be reachable by the collector). >>> >>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>> traced reference is stored (or might be stored, e.g., for VAR) >>> into a traced object. The check makes sure that the generational >>> GC knows that the object has been modified. It needs to know >>> this for objects in the older genration. >>> >>> Again, I reiterate that it is not my intention that programmers >>> actually call these runtime hooks explicitly! I am certain I can >>> rewrite any code you might have so that you do not need to call >>> them explicitly. Instead, properly typed references will result >>> in the checks being generated for you by the compiler. Please >>> feel free to send me code snippets for review. >>> >>>> I'm sure you haven't got time to go into the minutiae of the >>>> runtime system, but any hints would be appreciated. >>>> >>>> Cheers, >>>> Darko. >>>> >>>> >>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>> >>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>> This is wrong. When reading a REF field you should use the >>>>> following idiom: >>>>> >>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>> ... access field via ref^ ... >>>>> END; >>>>> >>>>> This will automatically insert a call to the appropriate >>>>> runtime routines on accessing the reference field. >>>>> >>>>> There should be no need for you to call the runtime routines >>>>> directly. >>>>> >>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Wondering if you can explain the use of these calls a little >>>>>> more. I'm currently using type maps to read and write fields >>>>>> from traced objects. Reading a traced reference from inside a >>>>>> traced object into a local variable is not working as it >>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>> objects means you need to call CheckStoreTraced on the object >>>>>> written inside of, before it is written? >>>>>> >>>>>> Cheers, >>>>>> Darko. >>>>> >>> >> > From dragisha at m3w.org Thu Apr 19 19:24:57 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 19:24:57 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's Message-ID: <1177003497.11459.17.camel@faramir.m3w.org> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box (Pentium M). It breaks somewhere where PklFonts is to be called. I am not sure if it's because of built binary being started w/ my glibc, or some other issue. Is cm3-5.4.0 possible to build on box like this? Thanks, dd -- Dragi?a Duri? From hosking at cs.purdue.edu Thu Apr 19 19:39:55 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:39:55 -0400 Subject: [M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced In-Reply-To: References: <00D31FAE-F446-4A35-96B9-DA922369A11D@darko.org> <7E0AE1FD-AB0B-45F3-8B58-3BADC7C667BC@darko.org> <3E0D85D8-FDF6-4C21-A8E5-0CB2055ECDA6@cs.purdue.edu> <3D3B31E7-CBA8-4FF7-B970-831EA093473D@darko.org> <061911BE-93F1-4B47-92A1-D30355CB7429@cs.purdue.edu> Message-ID: On Apr 19, 2007, at 1:09 PM, Darko wrote: > Thanks, thats very useful. Garbage collection is a bit of a > mystery, since I can't imagine myself changing it I've never > studied it, but it's on my mind constantly. M3 is so safe whenever > something weird happens with my code I know it's because I've > trodden on a traced reference, or copied it into a untraced space > and back or something. Yeah, it can be a problem, but if you are careful it should not be too hard to avoid. The idea is not to "hide" a reference where the GC cannot find it. > > I've relied on the pinning of references that appear in the stack > extensively, and I think a lot of other code does too. It a bit of > a worry to think that it's possible it might disappear. If we don't > have this, short of stopping the collector, what do you do if you > want your object to stay still? In the absence of implicit pinning of references from the stack we would need to introduce an explicit pin/unpin API for programmers to use. It is unlikely that CM3 will ever move away from implicit pinning unless we were to build a "smart" backend compiler (ie, not based on gcc). So, short answer is that you are probably OK for a long time to come. > > > On 19/04/2007, at 3:34 PM, Tony Hosking wrote: > >> My immediate reaction is that you probably want to approach reads >> slightly differently. Instead of using your generic InTypeRead >> routine to access the appropriate bits/bytes of the value, you >> probably want to replace it with some sort of simple addressing >> mechanism that computes the raw address of the field. Traced >> reference fields must always be aligned within the heap object >> they are stored in so your addressing for these will not need bit >> offsets. Once you have an ADDRESS "addr" for the traced reference >> field, so long as you use the idiom I gave earlier, you will get >> the correct behavior for reads. That is, >> >> WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO >> .. use field^ to access the traced reference stored at address >> field .. >> END; >> >> The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on >> the traced reference "r" once you have loopholed it through the >> raw field access into a REFANY, before you return it from >> InTypeReadRef: >> >> PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL): >> REFANY = >> VAR v: REFANY; >> BEGIN >> this.check(field, Kind.Ref, obj); >> this.read(obj, field, ADR(v), BYTESIZE(v)); >> RTHooks.CheckLoadTracedRef(v); >> RETURN v; >> END InTypeReadRef; >> >> This is safe for the current CM3 ambiguous roots collector, but >> this code would probably break with a fully copying/relocating >> collector (if one was ever implemented for Modula-3), since the >> raw bytes of the reference might become stale between the point >> you load them from memory and the point they are placed in the >> typed REFANY variable "v" (effectively, you'd be hiding a >> reference from the GC for a brief period, and it would not be able >> to update that hidden reference in the face of its target being >> moved). >> >> I note, generally, that your field addressing mechanisms only work >> anyway because we are using an ambiguous roots collector that pins >> all objects referenced from the thread stacks (your "obj" >> parameters in the accessor methods), which allows the use of >> RTHeap.GetDataAdr to provide an address for the object that is >> valid while the object reference is held on the stack. >> >> When writing references to object fields you will need to invoke >> "RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field >> is being stored. Thus, for InTypeWriteRef: >> >> PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field: >> CARDINAL) = >> BEGIN >> this.check(field, Kind.Ref, obj); >> WITH f = this.offs.get(field) DO >> IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE >> (val), NARROW(f, RefField).tc) THEN RAISE WrongType END; >> END; >> this.write(obj, field, ADR(val), BYTESIZE(val)); >> RTHooks.CheckStoreTraced(obj); >> END InTypeWriteRef; >> >> If you are performing multiple writes of reference fields on the >> same object (e.g., to an array) then you can invoke >> CheckStoreTraced once for the whole object. >> >> Hope this helps. >> >> On Apr 18, 2007, at 10:49 PM, Darko wrote: >> >>> Thanks, that's very helpful. I agree I shouldn't need it but it's >>> exactly because I'm not using the compiler is why I need to know >>> about it. The code is attached, it's still in development and >>> unfinished but you'll get the idea. >>> >>> Basically the module allows you to programatically access >>> structure fields. It allows for applications like being able to >>> create indexes (using an adaptation of the Table code) of objects >>> using arbitrary fields as keys. It makes dynamic programming a >>> bit more dynamic. The code is intended to be completely safe, but >>> there's more checks I need to do. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 19/04/2007, at 4:05 AM, Tony Hosking wrote: >>> >>>> >>>> On Apr 18, 2007, at 9:37 PM, Darko wrote: >>>> >>>>> Not actually using that code, but I did see your helpful commit >>>>> on it some time ago. It seems that the problem I described >>>>> earlier is due to another bug. The code I'm working on does a >>>>> lot of reading and writing of traced references and I was >>>>> hoping to get a better understanding of situations I need to be >>>>> aware of and how to use those RTHooks calls more efficiently. >>>> >>>> You SHOULD NOT need to use these calls so long as you are >>>> accessing using properly typed references (as in my example >>>> earlier). This is not an issue for SAFE code, only in UNSAFE >>>> modules must you be careful how you use LOOPHOLE to cast >>>> references. >>>> >>>>> So say I have two traced objects and I am copying a traced >>>>> reference from a field in one to the other. What set of calls >>>>> would be normally required there? >>>> >>>> How are you copying the references? I would like to see your >>>> code for this. So long as you properly type things then the >>>> calls to RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced >>>> will be generated by the compiler. >>>> >>>> Anyway... >>>> >>>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever >>>> you access a traced reference in memory (e.g., via dereference >>>> or from a shared variable). Its job is to prevent mutators from >>>> acquiring references to "gray" objects (that are reachable but >>>> still to be scanned for the traced references that they >>>> contain). The check turns the object from gray to black by >>>> scanning its references and making sure none of them refer to >>>> white objects (that are not known to be reachable by the >>>> collector). >>>> >>>> RTHooks.CheckStoreTraced is generated by the compiler whenever a >>>> traced reference is stored (or might be stored, e.g., for VAR) >>>> into a traced object. The check makes sure that the >>>> generational GC knows that the object has been modified. It >>>> needs to know this for objects in the older genration. >>>> >>>> Again, I reiterate that it is not my intention that programmers >>>> actually call these runtime hooks explicitly! I am certain I >>>> can rewrite any code you might have so that you do not need to >>>> call them explicitly. Instead, properly typed references will >>>> result in the checks being generated for you by the compiler. >>>> Please feel free to send me code snippets for review. >>>> >>>>> I'm sure you haven't got time to go into the minutiae of the >>>>> runtime system, but any hints would be appreciated. >>>>> >>>>> Cheers, >>>>> Darko. >>>>> >>>>> >>>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote: >>>>> >>>>>> I assume you have an apply method for your RTTypeMap.Visitor >>>>>> that takes "field: ADDRESS" and treats it as "REF REFANY". >>>>>> This is wrong. When reading a REF field you should use the >>>>>> following idiom: >>>>>> >>>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO >>>>>> ... access field via ref^ ... >>>>>> END; >>>>>> >>>>>> This will automatically insert a call to the appropriate >>>>>> runtime routines on accessing the reference field. >>>>>> >>>>>> There should be no need for you to call the runtime routines >>>>>> directly. >>>>>> >>>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Wondering if you can explain the use of these calls a little >>>>>>> more. I'm currently using type maps to read and write fields >>>>>>> from traced objects. Reading a traced reference from inside a >>>>>>> traced object into a local variable is not working as it >>>>>>> should. Should I use CheckLoadTraced and if so when and how? >>>>>>> Looking at your changes to RTTypeMap, writing references into >>>>>>> objects means you need to call CheckStoreTraced on the object >>>>>>> written inside of, before it is written? >>>>>>> >>>>>>> Cheers, >>>>>>> Darko. >>>>>> >>>> >>> >> From hosking at cs.purdue.edu Thu Apr 19 19:41:08 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 13:41:08 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177003497.11459.17.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: I just bootstrapped cm3 from the CVS head last night, on the most recent up-to-date Fedora core, so I don't know what your problems could be. I wonder if you have a broken bootstrap compiler? On Apr 19, 2007, at 1:24 PM, Dragi?a Duri? wrote: > I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > (Pentium > M). It breaks somewhere where PklFonts is to be called. I am not > sure if > it's because of built binary being started w/ my glibc, or some other > issue. > > Is cm3-5.4.0 possible to build on box like this? > > Thanks, > dd > > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Thu Apr 19 19:55:10 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Thu, 19 Apr 2007 19:55:10 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> Message-ID: <20070419175510.GA16758@elegosoft.com> On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > I just bootstrapped cm3 from the CVS head last night, on the most > recent up-to-date Fedora core, so I don't know what your problems > could be. I wonder if you have a broken bootstrap compiler? > > On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > > >I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >(Pentium > >M). It breaks somewhere where PklFonts is to be called. I am not > >sure if > >it's because of built binary being started w/ my glibc, or some other > >issue. > > > >Is cm3-5.4.0 possible to build on box like this? Hallo Antony, could you provide a minimal binary installation package for LINUXLIBC6 with PTHREADs that we could put on our server for download? I assume many people will run into problems on Linux with the current release. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Thu Apr 19 22:27:42 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 16:27:42 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419175510.GA16758@elegosoft.com> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: >> I just bootstrapped cm3 from the CVS head last night, on the most >> recent up-to-date Fedora core, so I don't know what your problems >> could be. I wonder if you have a broken bootstrap compiler? >> >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: >> >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box >>> (Pentium >>> M). It breaks somewhere where PklFonts is to be called. I am not >>> sure if >>> it's because of built binary being started w/ my glibc, or some >>> other >>> issue. >>> >>> Is cm3-5.4.0 possible to build on box like this? > > Hallo Antony, > > could you provide a minimal binary installation package for LINUXLIBC6 > with PTHREADs that we could put on our server for download? I assume > many people will run into problems on Linux with the current release. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Thu Apr 19 23:01:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 19 Apr 2007 23:01:20 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> Message-ID: <1177016480.4282.13.camel@faramir.m3w.org> I will try this one... But I am not sure we understand each other here... Bootstrap compiler from download site works for me, except for problems with PklFonts and mentor, also met by other ppl... I will look in that sometimes, as Trestle apps are not of primary interest for me anyway... Problem happens when trying to run some executable, I'll paste fisheye example... faramir:dragisha/pts/7: ~/cm3-inst% fisheye zsh: segmentation fault fisheye It is what happens. I must fix it with: faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye Is this only way to run cm3 built executables under modern Linuces? dd On Thu, 2007-04-19 at 16:27 -0400, Tony Hosking wrote: > There's a LINUXLIBC6 cm3 bootstrap compiler at ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > On Apr 19, 2007, at 1:55 PM, Olaf Wagner wrote: > > > On Thu, Apr 19, 2007 at 01:41:08PM -0400, Tony Hosking wrote: > >> I just bootstrapped cm3 from the CVS head last night, on the most > >> recent up-to-date Fedora core, so I don't know what your problems > >> could be. I wonder if you have a broken bootstrap compiler? > >> > >> On Apr 19, 2007, at 1:24 PM, Dragi??a Duri?? wrote: > >> > >>> I've tried to bootstrap latest cm3 (5.4.0) on fedora Core 6 box > >>> (Pentium > >>> M). It breaks somewhere where PklFonts is to be called. I am not > >>> sure if > >>> it's because of built binary being started w/ my glibc, or some > >>> other > >>> issue. > >>> > >>> Is cm3-5.4.0 possible to build on box like this? > > > > Hallo Antony, > > > > could you provide a minimal binary installation package for LINUXLIBC6 > > with PTHREADs that we could put on our server for download? I assume > > many people will run into problems on Linux with the current release. > > > > Olaf > > -- > > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > > Berlin, Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > > 23 45 86 95 > > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > > Sitz: Berlin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > > DE163214194 > -- Dragi?a Duri? From stsp at elego.de Thu Apr 19 23:05:09 2007 From: stsp at elego.de (Stefan Sperling) Date: Thu, 19 Apr 2007 23:05:09 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <1177016480.4282.13.camel@faramir.m3w.org> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> Message-ID: <20070419210509.GB11507@jack.stsp.lan> On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: > Problem happens when trying to run some executable, I'll paste fisheye > example... > > faramir:dragisha/pts/7: ~/cm3-inst% fisheye > zsh: segmentation fault fisheye > > It is what happens. I must fix it with: > > faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye > > Is this only way to run cm3 built executables under modern Linuces? Yes. It's because of a change in recent glibc versions. I forgot the specifics, I guess someone else will have to elaborate :-P -- stefan http://stsp.name PGP Key: 0xF59D25F0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: From hosking at cs.purdue.edu Thu Apr 19 23:44:20 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 19 Apr 2007 17:44:20 -0400 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: <20070419210509.GB11507@jack.stsp.lan> References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: Download and unzip the bootstrap compiler executable from ftp:// ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. Rebuild using this bootstrap compiler from the most recent CVS sources checked out from elegosoft. This will build a pthreads-enabled run-time system that avoids the pointer guard problems with user-threading based on setjmp/longjmp. I verified this build for me just last night on the most recent release of Fedora Core. On Apr 19, 2007, at 5:05 PM, Stefan Sperling wrote: > On Thu, Apr 19, 2007 at 11:01:20PM +0200, Dragi??a Duri?? wrote: >> Problem happens when trying to run some executable, I'll paste >> fisheye >> example... >> >> faramir:dragisha/pts/7: ~/cm3-inst% fisheye >> zsh: segmentation fault fisheye >> >> It is what happens. I must fix it with: >> >> faramir:dragisha/pts/7: ~/cm3-inst% LD_POINTER_GUARD=0 fisheye >> >> Is this only way to run cm3 built executables under modern Linuces? > > Yes. It's because of a change in recent glibc versions. > I forgot the specifics, I guess someone else will have to > elaborate :-P > -- > stefan > http://stsp.name PGP Key: > 0xF59D25F0 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at plane.elego.de Fri Apr 20 07:59:02 2007 From: wagner at plane.elego.de (Olaf Wagner) Date: Fri, 20 Apr 2007 07:59:02 +0200 Subject: [M3devel] Status of CM3 on LINUXLIBC6 with newer (LD_POINTER_GUARD...) glibc's In-Reply-To: References: <1177003497.11459.17.camel@faramir.m3w.org> <20070419175510.GA16758@elegosoft.com> <1177016480.4282.13.camel@faramir.m3w.org> <20070419210509.GB11507@jack.stsp.lan> Message-ID: <20070420055902.GA1418@elegosoft.com> On Thu, Apr 19, 2007 at 05:44:20PM -0400, Tony Hosking wrote: > Download and unzip the bootstrap compiler executable from ftp:// > ftp.cs.purdue.edu/pub/hosking/m3/LINUXLIBC6/cm3.gz. > > Rebuild using this bootstrap compiler from the most recent CVS > sources checked out from elegosoft. > > This will build a pthreads-enabled run-time system that avoids the > pointer guard problems with user-threading based on setjmp/longjmp. > > I verified this build for me just last night on the most recent > release of Fedora Core. Hi Stefan, I just checked our known problem pages, and the problem with the segment violation in indeed already documented. Could you additionally put up Antony's short description together with his link for a current CM3 bootstrap? Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 09:25:38 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 09:25:38 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. Message-ID: <1177053938.4282.29.camel@faramir.m3w.org> While maintaining our spinoff pm3 version, I've cleaned most of syscall wrapper needs from library source. Problem during early m3core development was, most probably, unclean situation about where and how C library will be invoked and someone decided to play safe, and to wrap all syscalls implied by used library calls to be fully safe. Across m3core and libm3, number of these low level, C library calls is really small. File I/O, socket I/O, filesystem operations... make a big majority of them (more than 98%). As all of these low level C lib calls these were easily identifiable (few grep's), I've cleaned them without much problems and after that, I've eliminated wrapper by wrapper. At this moment, I think none is present in our in-house m3 env. As time permits, and unless none else makes these steps, I will repeat them for CM3. I know answer to this following question is hidden somewhere in archives, but as my previous questions has shown - it can be hidden very deep :). So, can someone highlight current m3gc and syscall wrappers situation for me? How do I build CM3 WITH most extensive GC available at the moment? dd P.S. Big thanks to Tony Hosking for LD_POINTER_GUARD! As I have many Modula-3 apps in production use, this was very big issue for me. -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 10:07:20 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 10:07:20 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles Message-ID: <1177056440.4282.34.camel@faramir.m3w.org> while trying to make my programs with newsly installed cm3, I am constantly running into same error. Example: import("libm3") include_dir("../../../../m3lib0/src") include_dir("../../../../pdf/src") include_dir("../../../../zip/src") implementation("Test") program("test") Is my m3makefile, and when "cm3" is run in package's src dir, I have only: faramir:dragisha/pts/9: test/1/src% cm3 --- building in ../LINUXLIBC6 --- *** *** runtime error: *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES list *** file "../src/os/POSIX/PathnamePosix.m3", line 98 *** Mentioned line raises this FATAL exception when Absolute(path) is TRUE... What is absolute in my paths? All m3makefiles in question were ok with pm3, of course.. dd -- Dragi?a Duri? From wagner at elegosoft.com Fri Apr 20 11:57:34 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 11:57:34 +0200 (CEST) Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <1177053938.4282.29.camel@faramir.m3w.org> References: <1177053938.4282.29.camel@faramir.m3w.org> Message-ID: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: > While maintaining our spinoff pm3 version, I've cleaned most of syscall > wrapper needs from library source. Problem during early m3core > development was, most probably, unclean situation about where and how C > library will be invoked and someone decided to play safe, and to wrap > all syscalls implied by used library calls to be fully safe. > > Across m3core and libm3, number of these low level, C library calls is > really small. File I/O, socket I/O, filesystem operations... make a big > majority of them (more than 98%). As all of these low level C lib calls > these were easily identifiable (few grep's), I've cleaned them without > much problems and after that, I've eliminated wrapper by wrapper. At > this moment, I think none is present in our in-house m3 env. You would also need to consider all indirect system calls via libraries and embedded C code. I think the M3 developers tried to offer a more or less complete set of safe system calls, regardless how they are called from the M3 program. > As time > permits, and unless none else makes these steps, I will repeat them for > CM3. I don't think this is a good idea right now. > I know answer to this following question is hidden somewhere in > archives, but as my previous questions has shown - it can be hidden very > deep :). So, can someone highlight current m3gc and syscall wrappers > situation for me? How do I build CM3 WITH most extensive GC available at > the moment? The current CM3 compiler/runtime does not need virtual memory protection and thus does not need the system call wrappers any more, as Antony has modified the compiler to generate hints for the garbage collector. So you can more or less ignore this issue (unless you really want memory protection) and simply use m3gc-simple for all platforms. Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 14:21:19 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:19 +0200 Subject: [M3devel] cm3 pthread problem? Message-ID: <1177071680.4282.47.camel@faramir.m3w.org> Attached program won't run under freshly built cm3 @ Fedora Core 6. Compiles, runs... and stucks... It works flawlessly with user-level threads and also with mine implementation of NPTL for pm3. I've done it for 4000 threads per 10000 passess, and this version I am attaching, and trying without success on CM3 is 40 threads... Of course, before I run it, I am ulimiting stack size to 64kB or similar.. It fits. dd -- Dragi?a Duri? From dragisha at m3w.org Fri Apr 20 14:21:56 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 14:21:56 +0200 Subject: [M3devel] missing attachment, sorry Message-ID: <1177071717.4282.49.camel@faramir.m3w.org> -- Dragi?a Duri? -------------- next part -------------- MODULE TestThreads EXPORTS Main; IMPORT Fmt, IO, Thread; TYPE MyClosure = Thread.Closure OBJECT my, next: Thread.Condition; OVERRIDES apply := JustDoIt; END; VAR glob := NEW(MUTEX); started: CARDINAL; PROCEDURE JustDoIt(c: MyClosure): REFANY = BEGIN LOCK glob DO INC(started); Thread.Wait(glob, c.my); Thread.Signal(c.next); END; RETURN NIL; END JustDoIt; CONST nThreads = 40; VAR my, first, a, b: Thread.Condition; BEGIN FOR j:=1 TO 10 DO IO.Put("Forking " & Fmt.Int(nThreads) & " threads... (" & Fmt.Int(j) & ")\n"); IF j MOD 10 = 0 THEN IO.Put("(" & Fmt.Int(j) & ")"); END; my := NEW(Thread.Condition); a:=NEW(Thread.Condition); started:=0; FOR i:=1 TO nThreads DO IF i=nThreads THEN b:=my; ELSE b:=NEW(Thread.Condition); END; EVAL Thread.Fork(NEW(MyClosure, my := a, next := b)); IF i=1 THEN first:=a; END; a:=b; END; IO.Put("Forked.\n"); IO.Put(","); WHILE started References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> Message-ID: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 1:10 pm, Dragi??a Duri?? wrote: > On Fri, 2007-04-20 at 11:57 +0200, Olaf Wagner wrote: >> On Fri, April 20, 2007 9:25 am, Dragi? ??a Duri????? wrote: >> > While maintaining our spinoff pm3 version, I've cleaned most of syscall >> > wrapper needs from library source. Problem during early m3core >> > development was, most probably, unclean situation about where and how C >> > library will be invoked and someone decided to play safe, and to wrap >> > all syscalls implied by used library calls to be fully safe. >> > >> > Across m3core and libm3, number of these low level, C library calls is >> > really small. File I/O, socket I/O, filesystem operations... make a big >> > majority of them (more than 98%). As all of these low level C lib calls >> > these were easily identifiable (few grep's), I've cleaned them without >> > much problems and after that, I've eliminated wrapper by wrapper. At >> > this moment, I think none is present in our in-house m3 env. >> >> You would also need to consider all indirect system calls via libraries >> and embedded C code. I think the M3 developers tried to offer a more or >> less complete set of safe system calls, regardless how they are called >> from the M3 program. > > It is, but C libraries are mostly in these few percents remaining... > Good wrapping techniques and auditing of existing libraries (hint: grep > EXTERNAL) would be small effort for seamless integration of full m3 > runtime with unperfect C world. I was trying to say that cover only the calls from libraries now used by the M3 runtime, but M3 is an open language and allows the integration of C and C++ (and other) code (and thus of arbitrary other libraries). >> > I know answer to this following question is hidden somewhere in >> > archives, but as my previous questions has shown - it can be hidden very >> > deep :). So, can someone highlight current m3gc and syscall wrappers >> > situation for me? How do I build CM3 WITH most extensive GC available at >> > the moment? >> >> The current CM3 compiler/runtime does not need virtual memory protection >> and thus does not need the system call wrappers any more, as Antony has >> modified the compiler to generate hints for the garbage collector. So >> you can more or less ignore this issue (unless you really want memory >> protection) and simply use m3gc-simple for all platforms. > > Does it mean garbage collection is not > incremental/generational/supercallifraglistic... anymore, or Anthony's > work has made it work without memory protection? The latter; we've got incremental and generational garbage collection without the need for memory protection and system call wrappers :) Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Fri Apr 20 14:59:32 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 08:59:32 -0400 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> Message-ID: <9B2640F3-10E7-4CE6-BB63-32090198FA1C@cs.purdue.edu> Yes, I concur that there is no need to expend any effort on syscall wrappers. I plan to eliminate the legacy syscall wrappers from all CM3 targets very soon, since the new GC no longer needs them. I would hate to see the library sources cluttered with unnecessary changes that relate only to the deprecated syscall wrappers. On Apr 20, 2007, at 5:57 AM, Olaf Wagner wrote: > > On Fri, April 20, 2007 9:25 am, Dragi??a Duri?? wrote: >> While maintaining our spinoff pm3 version, I've cleaned most of >> syscall >> wrapper needs from library source. Problem during early m3core >> development was, most probably, unclean situation about where and >> how C >> library will be invoked and someone decided to play safe, and to wrap >> all syscalls implied by used library calls to be fully safe. >> >> Across m3core and libm3, number of these low level, C library >> calls is >> really small. File I/O, socket I/O, filesystem operations... make >> a big >> majority of them (more than 98%). As all of these low level C lib >> calls >> these were easily identifiable (few grep's), I've cleaned them >> without >> much problems and after that, I've eliminated wrapper by wrapper. At >> this moment, I think none is present in our in-house m3 env. > > You would also need to consider all indirect system calls via > libraries > and embedded C code. I think the M3 developers tried to offer a > more or > less complete set of safe system calls, regardless how they are called > from the M3 program. > >> As time >> permits, and unless none else makes these steps, I will repeat >> them for >> CM3. > > I don't think this is a good idea right now. > >> I know answer to this following question is hidden somewhere in >> archives, but as my previous questions has shown - it can be >> hidden very >> deep :). So, can someone highlight current m3gc and syscall wrappers >> situation for me? How do I build CM3 WITH most extensive GC >> available at >> the moment? > > The current CM3 compiler/runtime does not need virtual memory > protection > and thus does not need the system call wrappers any more, as Antony > has > modified the compiler to generate hints for the garbage collector. So > you can more or less ignore this issue (unless you really want memory > protection) and simply use m3gc-simple for all platforms. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Fri Apr 20 15:06:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 15:06:11 +0200 Subject: [M3devel] syscall wrappers, m3gc and cm3.. In-Reply-To: <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> References: <1177053938.4282.29.camel@faramir.m3w.org> <63048.194.138.127.36.1177063054.squirrel@mail.elegosoft.com> <1177067409.4282.42.camel@faramir.m3w.org> <56434.194.138.127.36.1177073792.squirrel@mail.elegosoft.com> Message-ID: <1177074372.4282.54.camel@faramir.m3w.org> On Fri, 2007-04-20 at 14:56 +0200, Olaf Wagner wrote: ... > >> You would also need to consider all indirect system calls via libraries > >> and embedded C code. I think the M3 developers tried to offer a more or > >> less complete set of safe system calls, regardless how they are called > >> from the M3 program. > > > > It is, but C libraries are mostly in these few percents remaining... > > Good wrapping techniques and auditing of existing libraries (hint: grep > > EXTERNAL) would be small effort for seamless integration of full m3 > > runtime with unperfect C world. > > I was trying to say that cover only the calls from libraries now used > by the M3 runtime, but M3 is an open language and allows the integration > of C and C++ (and other) code (and thus of arbitrary other libraries). It is exactly what I was pointing to... Arbitrary other libraries will have defined entry points from Modula-3 code, and that is what matters... What and how C library code is doing with _its data_ is not of our concern. What is it doing with passed data concerns us, and it is a place where interfacing guidelines/policies kick in. ... > > Does it mean garbage collection is not > > incremental/generational/supercallifraglistic... anymore, or Anthony's > > work has made it work without memory protection? > > The latter; we've got incremental and generational garbage collection > without the need for memory protection and system call wrappers :) Oh. Oh. Ohhhhhhhhh. :) GREAT! > > Olaf -- Dragi?a Duri? From hosking at cs.purdue.edu Fri Apr 20 15:08:51 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:08:51 -0400 Subject: [M3devel] cm3 pthread problem? In-Reply-To: <1177071680.4282.47.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> Message-ID: I'm investigating this issue. Looks like some sort of race causing deadlock. On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: > Attached program won't run under freshly built cm3 @ Fedora Core 6. > Compiles, runs... and stucks... > > It works flawlessly with user-level threads and also with mine > implementation of NPTL for pm3. I've done it for 4000 threads per > 10000 > passess, and this version I am attaching, and trying without > success on > CM3 is 40 threads... > > Of course, before I run it, I am ulimiting stack size to 64kB or > similar.. It fits. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hosking at cs.purdue.edu Fri Apr 20 15:16:33 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 20 Apr 2007 09:16:33 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <1177056440.4282.34.camel@faramir.m3w.org> References: <1177056440.4282.34.camel@faramir.m3w.org> Message-ID: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> I have my hands full with GC evolution and thread issues at the moment. Anyone else care to look into this? On Apr 20, 2007, at 4:07 AM, Dragi?a Duri? wrote: > while trying to make my programs with newsly installed cm3, I am > constantly running into same error. Example: > > import("libm3") > > include_dir("../../../../m3lib0/src") > include_dir("../../../../pdf/src") > include_dir("../../../../zip/src") > > implementation("Test") > program("test") > > Is my m3makefile, and when "cm3" is run in package's src dir, I have > only: > > faramir:dragisha/pts/9: test/1/src% cm3 > --- building in ../LINUXLIBC6 --- > > > > *** > *** runtime error: > *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > list > *** file "../src/os/POSIX/PathnamePosix.m3", line 98 > *** > > Mentioned line raises this FATAL exception when Absolute(path) is > TRUE... What is absolute in my paths? > > All m3makefiles in question were ok with pm3, of course.. > > dd > -- > Dragi?a Duri? > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at elegosoft.com Fri Apr 20 15:38:53 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 20 Apr 2007 15:38:53 +0200 (CEST) Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <57472.194.138.127.36.1177076333.squirrel@mail.elegosoft.com> On Fri, April 20, 2007 3:16 pm, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I can take this one, but won't be able to look into it until tonight (no M3 here at my current customer work), and if I don't find it fast, it will be delayed to Monday evening. If this is too late or anybody else can be faster, please let me know. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > >> while trying to make my programs with newsly installed cm3, I am >> constantly running into same error. Example: >> >> import("libm3") >> >> include_dir("../../../../m3lib0/src") >> include_dir("../../../../pdf/src") >> include_dir("../../../../zip/src") >> >> implementation("Test") >> program("test") >> >> Is my m3makefile, and when "cm3" is run in package's src dir, I have >> only: >> >> faramir:dragisha/pts/9: test/1/src% cm3 >> --- building in ../LINUXLIBC6 --- >> >> >> >> *** >> *** runtime error: >> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >> list >> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >> *** >> >> Mentioned line raises this FATAL exception when Absolute(path) is >> TRUE... What is absolute in my paths? >> >> All m3makefiles in question were ok with pm3, of course.. >> >> dd >> -- >> Dragi??a Duri?? >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Fri Apr 20 21:36:25 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 20 Apr 2007 21:36:25 +0200 Subject: [M3devel] order of module initialization... Message-ID: <1177097786.4282.75.camel@faramir.m3w.org> I remember debugging this first time we tried to migrate to CM3... And then we solved it, for current CM3 and current version of our software, mentioning some imported module in EXPORTS clause... at the moment, this linked these modules in initialization chain and it worked ok... Right now, it does not. Module XLBuiltin is one imported regularly, and XLModuleUDP (for example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ wrong moment... What would be solution for this? dd -- Dragi?a Duri? From jayk123 at hotmail.com Fri Apr 20 23:58:55 2007 From: jayk123 at hotmail.com (j k) Date: Fri, 20 Apr 2007 21:58:55 +0000 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: Reminds me of C++ globals with constructors. The solution is to avoid initialization. Do everything "on demand" and hide all state behind functions. and then "on demand" needs to be thread safe, so there is a small bootstrapping problem -- basically the only initialization you should do, speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more accurately, nothing "in the same layer", if you have a dependency graph, only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are (unless you are working on ntdll.dll, but even then.. :) ) If you don't have a strict layered design, or can't capture what it is and analyze statically, again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an easy way to be correct. And even then, you can often avoid the critical sections via something like InterlockedCompareExchangePointer, even implement spin locks for this rare code. (Hand written spin locks are generally to be avoided since the kernel won't know about them and you won't schedule efficiently, whereas with EnterCriticalSection, if there is contention, the kernel has a hint what is going on and can schedule better). Don't have any "public global data", unless, and I don't know if this applies in Modula-3, unless it is "constant initialized", which I'm not sure is well defined in C++ or not. I do believe it is well defined in C..except dynamic linking I think screws that up to. (pardon the Win32-centricity; I'm sure the principles apply more broadly) Ok, in C++ and in Modula-3, there are maybe better answers. In C++, globals within a source file are guaranteed to be initialized in the order they occur in the file. But the order of files is not defined. There is a documented trick..potentially very inefficient..that I don't have the patience to describe here.. "the iostream counter trick"... this I don't think has an analog in Modula-3, but sure. Stroupstroup's Design and Evolution book documents it. - Jay >From: Dragi??a Duri?? >To: "M3 Developers' Mailing List" >Subject: [M3devel] order of module initialization... >Date: Fri, 20 Apr 2007 21:36:25 +0200 > >I remember debugging this first time we tried to migrate to CM3... And >then we solved it, for current CM3 and current version of our software, >mentioning some imported module in EXPORTS clause... at the moment, this >linked these modules in initialization chain and it worked ok... Right >now, it does not. > >Module XLBuiltin is one imported regularly, and XLModuleUDP (for >example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >wrong moment... > >What would be solution for this? > >dd >-- >Dragi??a Duri?? > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _________________________________________________________________ Download Messenger. Join the i?m Initiative. Help make a difference today. http://im.live.com/messenger/im/home/?source=TAGHM_APR07 From mika at async.caltech.edu Sat Apr 21 00:15:16 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 20 Apr 2007 15:15:16 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Fri, 20 Apr 2007 21:58:55 -0000." Message-ID: <200704202215.l3KMFGY4013501@camembert.async.caltech.edu> The order in Modula-3 is defined in section 2.5.6 of the language report (page 47 of Systems Programming with Modula-3). The problem described here appears to be that EXPORTS isn't treated the same as IMPORT (which it should be). There are some other problems I noticed when porting PM3 code to CM3, don't know if they have been fixed. The "solution" in that case was to IMPORT interfaces in places where they weren't really needed. Obfuscating one's code to avoid taking advantage of the convenience of module initialization seems like a rather severe suggestion... The rule, by the way, is: "If module M depends on module N and N does not depend on M, then N's body will be executed before M's body, where: * A module M 'depends on' a module N if M uses an interface that N exports or if M depends on a module that depends on N. * A module M 'uses' an interface I if M imports or exports I or if M uses an interface that imports I. Except for this constraint, the order of execution is implementation-dependent." Mika "j k" writes: >Reminds me of C++ globals with constructors. > >The solution is to avoid initialization. > >Do everything "on demand" and hide all state behind functions. > and then "on demand" needs to be thread safe, so there is a small >bootstrapping problem -- basically the only initialization you should do, >speaking in Win32-ese, is a few calls to InitializeCriticalSection and maybe >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >accurately, nothing "in the same layer", if you have a dependency graph, >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always are >(unless you are working on ntdll.dll, but even then.. :) ) If you don't have >a strict layered design, or can't capture what it is and analyze statically, >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is an >easy way to be correct. > >And even then, you can often avoid the critical sections via something like >InterlockedCompareExchangePointer, even implement spin locks for this rare >code. (Hand written spin locks are generally to be avoided since the kernel >won't know about them and you won't schedule efficiently, whereas with >EnterCriticalSection, if there is contention, the kernel has a hint what is >going on and can schedule better). > >Don't have any "public global data", unless, and I don't know if this >applies in Modula-3, unless it is "constant initialized", which I'm not sure >is well defined in C++ or not. I do believe it is well defined in C..except >dynamic linking I think screws that up to. > >(pardon the Win32-centricity; I'm sure the principles apply more broadly) > >Ok, in C++ and in Modula-3, there are maybe better answers. >In C++, globals within a source file are guaranteed to be initialized in the >order they occur in the file. >But the order of files is not defined. >There is a documented trick..potentially very inefficient..that I don't have >the patience to describe here.. "the iostream counter trick"... this I don't >think has an analog in Modula-3, but sure. >Stroupstroup's Design and Evolution book documents it. > >- Jay > > >>From: Dragi??a Duri?? >>To: "M3 Developers' Mailing List" >>Subject: [M3devel] order of module initialization... >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >>I remember debugging this first time we tried to migrate to CM3... And >>then we solved it, for current CM3 and current version of our software, >>mentioning some imported module in EXPORTS clause... at the moment, this >>linked these modules in initialization chain and it worked ok... Right >>now, it does not. >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >>wrong moment... >> >>What would be solution for this? >> >>dd >>-- >>Dragi??a Duri?? >> >>_______________________________________________ >>M3devel mailing list >>M3devel at elegosoft.com >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Download Messenger. Join the i?m Initiative. Help make a difference today. >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dragisha at m3w.org Sun Apr 22 09:29:11 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 22 Apr 2007 09:29:11 +0200 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> Message-ID: <1177226951.737.1.camel@faramir.m3w.org> It looks like this fix never came through... With cvs-head CM3, unpickling of TextRefTbl.T instance still does not work. dd On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: > Rodney, > > I've checked in fixes to RTTypeMap and clients that should restore > functional pickling support. For some reason my commit messages are > awaiting moderator approval (it seems my commit id is not the same as > my subscription to the m3commit list) before they can be sent out. > Please make sure that unpickling now works for you. > > Best regards, > > Tony > > On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: > > > I tracked down a bug unpickling to the following. When executing > > in ConvertPacking.Read, at ConvertPacking.m3:327, this code: > > > > WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO > > ref^ := v.readRef(nelem.refType); > > END; > > > > has a compiler-generated call on RTCollector.CheckStoreTraced(ref). > > CheckStoreTraced expects ref to be a normal reference to an object, > > computes the address of its header (4 bytes less), and sets the > > dirty bit. > > > > But ConvertPacking is reading in to a field of a recently created > > object that is not first in the object. It computes ref to point > > to the field, and CheckStoreTraced actually steps on the previous > > real field instead of the header. > > > > It looks like the actions of CheckStoreTraced are needed, but I > > don't right off hand see how to recode Convert to fix this. It's > > use of ref can't be moved. Presumably, there could be other unsafe > > code elsewhere using the same technique to store data. > > > > Does the compiler need a different criterion on when to generate > > the call on CheckStoreTraced? ref is an UNTRACED REF here, but it > > still is pointing inside a traced object. Maybe people who modify > > objects in this way need to explicitly call CheckStoreTraced? > > > > -- > > ------------------------------------------------------------- > > Rodney M. Bates, retired assistant professor > > Dept. of Computer Science, Wichita State University > > Wichita, KS 67260-0083 > > 316-978-3922 > > rodney.bates at wichita.edu > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > Antony Hosking | Associate Professor > Dept of Computer Science | Office: +1 765 494-6001 > Purdue University | Mobile: +1 765 427-5484 > 250 N. University Street | Email: hosking at cs.purdue.edu > West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking > _--_|\ > / \ > \_.--._/ ) > v / > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:22:11 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:22:11 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <1177226951.737.1.camel@faramir.m3w.org> References: <45970723.5070102@wichita.edu> <924B960D-3F3C-419B-A7D3-3FB74B21C5CC@cs.purdue.edu> <1177226951.737.1.camel@faramir.m3w.org> Message-ID: Yes, this fix is in CVS head. On Apr 22, 2007, at 3:29 AM, Dragi?a Duri? wrote: > It looks like this fix never came through... With cvs-head CM3, > unpickling of TextRefTbl.T instance still does not work. > > dd > > On Tue, 2007-01-09 at 13:06 -0500, Tony Hosking wrote: >> Rodney, >> >> I've checked in fixes to RTTypeMap and clients that should restore >> functional pickling support. For some reason my commit messages are >> awaiting moderator approval (it seems my commit id is not the same as >> my subscription to the m3commit list) before they can be sent out. >> Please make sure that unpickling now works for you. >> >> Best regards, >> >> Tony >> >> On Dec 30, 2006, at 7:41 PM, Rodney M. Bates wrote: >> >>> I tracked down a bug unpickling to the following. When executing >>> in ConvertPacking.Read, at ConvertPacking.m3:327, this code: >>> >>> WITH ref = LOOPHOLE(dest, UNTRACED REF REFANY) DO >>> ref^ := v.readRef(nelem.refType); >>> END; >>> >>> has a compiler-generated call on RTCollector.CheckStoreTraced(ref). >>> CheckStoreTraced expects ref to be a normal reference to an object, >>> computes the address of its header (4 bytes less), and sets the >>> dirty bit. >>> >>> But ConvertPacking is reading in to a field of a recently created >>> object that is not first in the object. It computes ref to point >>> to the field, and CheckStoreTraced actually steps on the previous >>> real field instead of the header. >>> >>> It looks like the actions of CheckStoreTraced are needed, but I >>> don't right off hand see how to recode Convert to fix this. It's >>> use of ref can't be moved. Presumably, there could be other unsafe >>> code elsewhere using the same technique to store data. >>> >>> Does the compiler need a different criterion on when to generate >>> the call on CheckStoreTraced? ref is an UNTRACED REF here, but it >>> still is pointing inside a traced object. Maybe people who modify >>> objects in this way need to explicitly call CheckStoreTraced? >>> >>> -- >>> ------------------------------------------------------------- >>> Rodney M. Bates, retired assistant professor >>> Dept. of Computer Science, Wichita State University >>> Wichita, KS 67260-0083 >>> 316-978-3922 >>> rodney.bates at wichita.edu >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> Antony Hosking | Associate Professor >> Dept of Computer Science | Office: +1 765 494-6001 >> Purdue University | Mobile: +1 765 427-5484 >> 250 N. University Street | Email: hosking at cs.purdue.edu >> West Lafayette, IN 47907-2066 | http://www.cs.purdue.edu/~hosking >> _--_|\ >> / \ >> \_.--._/ ) >> v / >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:47:50 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:47:50 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177233124.737.4.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> Message-ID: <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> It is a pthreads call that is failing: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; I'm guessing we are in a race with a thread that is in the middle of dying (i.e., the error code, "r", is "ESRCH: thread is an invalid thread ID"). Can you confirm that is the error? As for why it is happening I don't know, since the thread handle is only ever accessed while activeMu is locked, which enforces atomicity of the thread's presence in the ring of live threads along with its handle being valid: WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; IF allThreads = me THEN allThreads := me.next; END; me.next.prev := me.prev; me.prev.next := me.next; me.next := NIL; me.prev := NIL; WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; I wonder if this is a system limit on the number of threads or some such (but then why would the pthread_create call not return an error?) Can you diagnose further? On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: > After upping it a bit, with stack limit set to 64k and 4000 > threads in > each loop... Is not happening at same place every run, but most of > runs > - it happens. > > Forking 4000 threads... (1) > Forked. > ,: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 > *** > > > On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >> I have fixed this bug and checked in the fix to CVS (why are we not >> seeing the commit messages again?). It did turn out to be a deadlock >> issue in the handshake between allocating threads and the GC. Your >> sample program now runs with no problems on my dual-core Intel Mac. >> Please let me know if you have any other threading difficulties. >> >> Regards, >> >> Tony >> >> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >> >>> Attached program won't run under freshly built cm3 @ Fedora Core 6. >>> Compiles, runs... and stucks... >>> >>> It works flawlessly with user-level threads and also with mine >>> implementation of NPTL for pm3. I've done it for 4000 threads per >>> 10000 >>> passess, and this version I am attaching, and trying without >>> success on >>> CM3 is 40 threads... >>> >>> Of course, before I run it, I am ulimiting stack size to 64kB or >>> similar.. It fits. >>> >>> dd >>> -- >>> Dragi?a Duri? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 21:59:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 15:59:46 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177253233.737.18.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> Message-ID: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > Hello, > > Have been skimming through source of PTHREAD code... And I think > job can > be done without so much relying on how-they-did-it-before, esp with > regard to list of waiters and similar internal and global structures. > Also, I see number of global locks and I am sure they are congestion > generators every now and while, esp in heavy threading situations. > > Of course, there is number of approaches to this multi-thread > situations. Mine being one of very nonconservative use of threads, I > think it is important to remain open to possibly very big number of > threads running in single process - meaning scalability is one of > primary objections... As global locks don't do well with > scalability, I > don't like "cm" and similar global congestion points. Yes, there are tensions between a thin/absent veneer between language threads and system threads. Most important are issues of preserving a reasonable memory model for programmers (see Hans Boehm's paper http://portal.acm.org/citation.cfm?doid=1065010.1065042). There are also questions of portability and debuggability. I agree that global locks are to be avoided where they cause known contention, but there are tradeoffs there too. For large numbers of threads (as you appear to need) I think we would need to adopt some other implementation approach, possibly by multiplexing multiple lightweight user-level threads on some smaller number of heavyweight system-level threads, but then you run into scheduling and load-balancing problems. > We talked about this at least once before, and I think I remember you > insisted on more compatibility than can be read from SPwM3. Do you > think > best idea would be to integrate mine NPTL code into CM3 for people to > trash and test, and let everyone select what is best for their > situation? What I wanted was a situation where programs would be able to run with the same tools (e.g., showheap, showthread) under both user- level and system threading. This goal has been achieved with the current pthread-based implementation. Moreover, I wanted something where variations in thread support from one system to another could be exploited most easily (such as for systems where thread suspend/ resume is provided as a primitive). Again, the current implementation achieves this, and runs with minimal target-specific code on Darwin, Solaris, and Linux. Ports to other targets should be relatively straightforward. > Problems I had with my pthread implementation were all related to VM > hell of earlier GC implementation... After you did that piece of art > with new approach to GC, I expect infinite uptimes from my servers and > bots :). Big thanks for that! Any native threading implementation is going to have problems with VM- based memory management. I'm surprised to were able to get anything going with the VM-based GC. > > dd > -- > Dragi?a Duri? From hosking at cs.purdue.edu Sun Apr 22 23:50:46 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 17:50:46 -0400 Subject: [M3devel] cm3 pthread problem?... another one? In-Reply-To: <1177275494.737.37.camel@faramir.m3w.org> References: <1177071680.4282.47.camel@faramir.m3w.org> <003CE8FA-EBA9-4534-AF49-92420BFD0CF2@cs.purdue.edu> <1177233124.737.4.camel@faramir.m3w.org> <2B1CD192-9DBE-40AD-AE3C-416452099C79@cs.purdue.edu> <1177275494.737.37.camel@faramir.m3w.org> Message-ID: <8C54780A-5D27-4239-92BE-1D274340BE5A@cs.purdue.edu> Argh! That signal isn't even documented as being returned by pthread_kill on Linux (at least in my man pages). So the solution appears to be to replace: WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO <*ASSERT r=0*> END; with: LOOP WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO IF r = 0 THEN EXIT; END; IF r # Uerror.EAGAIN THEN <*ASSERT FALSE*>; END; (* try it again *) END; END; What do you think? I wonder how many other places I should be checking for EAGAIN? On Apr 22, 2007, at 4:58 PM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:47 -0400, Tony Hosking wrote: >> It is a pthreads call that is failing: >> >> WITH r = Upthread.kill(act.handle, SIG_SUSPEND) DO >> <*ASSERT r=0*> >> END; >> >> I'm guessing we are in a race with a thread that is in the middle of >> dying (i.e., the error code, "r", is "ESRCH: thread is an invalid >> thread ID"). Can you confirm that is the error? > > No :). It's 11 - EAGAIN, IIRC. Lot's of signals are sent there, and it > kicked some signal dequeuing limit, or something like. > >> As for why it is happening I don't know, since the thread handle is >> only ever accessed while activeMu is locked, which enforces atomicity >> of the thread's presence in the ring of live threads along with its >> handle being valid: >> >> WITH r = Upthread.mutex_lock(activeMu) DO <*ASSERT r=0*> END; >> IF allThreads = me THEN allThreads := me.next; END; >> me.next.prev := me.prev; >> me.prev.next := me.next; >> me.next := NIL; >> me.prev := NIL; >> WITH r = Upthread.detach(me.handle) DO <*ASSERT r=0*> END; >> WITH r = Upthread.mutex_unlock(activeMu) DO <*ASSERT r=0*> END; >> >> I wonder if this is a system limit on the number of threads or some >> such (but then why would the pthread_create call not return an >> error?) Can you diagnose further? > > Limit is not, it's sure. I am running test in parallel with mine > implementation + pm3, and it works without a flaw, 10000 rounds, 4000 > threads each. I am using some realtime signal for suspend magic, > though... Don't remember right now, but I think there is something > special about this queuing of realtime and non-realtime signals... > >> >> On Apr 22, 2007, at 5:12 AM, Dragi?a Duri? wrote: >> >>> After upping it a bit, with stack limit set to 64k and 4000 >>> threads in >>> each loop... Is not happening at same place every run, but most of >>> runs >>> - it happens. >>> >>> Forking 4000 threads... (1) >>> Forked. >>> ,: >>> >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 1083 >>> *** >>> >>> >>> On Sat, 2007-04-21 at 10:48 -0400, Tony Hosking wrote: >>>> I have fixed this bug and checked in the fix to CVS (why are we not >>>> seeing the commit messages again?). It did turn out to be a >>>> deadlock >>>> issue in the handshake between allocating threads and the GC. Your >>>> sample program now runs with no problems on my dual-core Intel Mac. >>>> Please let me know if you have any other threading difficulties. >>>> >>>> Regards, >>>> >>>> Tony >>>> >>>> On Apr 20, 2007, at 8:21 AM, Dragi?a Duri? wrote: >>>> >>>>> Attached program won't run under freshly built cm3 @ Fedora >>>>> Core 6. >>>>> Compiles, runs... and stucks... >>>>> >>>>> It works flawlessly with user-level threads and also with mine >>>>> implementation of NPTL for pm3. I've done it for 4000 threads per >>>>> 10000 >>>>> passess, and this version I am attaching, and trying without >>>>> success on >>>>> CM3 is 40 threads... >>>>> >>>>> Of course, before I run it, I am ulimiting stack size to 64kB or >>>>> similar.. It fits. >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 00:00:03 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:00:03 -0400 Subject: [M3devel] Fwd: A GC/compiler/Pickle interaction bug References: <1177277134.737.47.camel@faramir.m3w.org> Message-ID: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> This works just fine for me on the most recent CVS head for I386_DARWIN. I wonder if you have not updated your compiler to match? There was a matching fix to the compiler. You should build and ship: m3gc-simple m3core libm3 m3middle m3linker m3front m3quake cm3 Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your cm3.cfg). This will get you a new compiler. Then, rebuild your CM3 installation from the CVS head. Begin forwarded message: > From: Dragi?a Duri? > Date: April 22, 2007 5:25:33 PM EDT > To: Tony Hosking > Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug > > On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >> Can you send me a stack dump from gdb at the point of the error? >> Even better, do you have a simple example program I can use to test >> with? > > Here is one. Quick and dirty, but it shows problem. > > -- > Dragi?a Duri? -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Pickles.m3 URL: -------------- next part -------------- From hosking at cs.purdue.edu Mon Apr 23 00:01:05 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 22 Apr 2007 18:01:05 -0400 Subject: [M3devel] A GC/compiler/Pickle interaction bug In-Reply-To: <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> References: <1177277134.737.47.camel@faramir.m3w.org> <9CC59642-6982-4612-BB7E-AE887152D89D@cs.purdue.edu> Message-ID: <19A8E27F-BCBE-4702-8BD0-EE2342D3D078@cs.purdue.edu> PS The bootstrap compilers on elegosoft probably may to be rebuilt for this fix to take effect. On Apr 22, 2007, at 6:00 PM, Tony Hosking wrote: > This works just fine for me on the most recent CVS head for > I386_DARWIN. > > I wonder if you have not updated your compiler to match? There was > a matching fix to the compiler. You should build and ship: > > m3gc-simple > m3core > libm3 > m3middle > m3linker > m3front > m3quake > cm3 > > Install cm3/TARGET/cm3 into your BIN_INSTALL (as defined by your > cm3.cfg). This will get you a new compiler. > > Then, rebuild your CM3 installation from the CVS head. > > Begin forwarded message: > >> From: Dragi?a Duri? >> Date: April 22, 2007 5:25:33 PM EDT >> To: Tony Hosking >> Subject: Re: [M3devel] A GC/compiler/Pickle interaction bug >> >> On Sun, 2007-04-22 at 16:36 -0400, Tony Hosking wrote: >>> Can you send me a stack dump from gdb at the point of the error? >>> Even better, do you have a simple example program I can use to test >>> with? >> >> Here is one. Quick and dirty, but it shows problem. >> >> -- >> Dragi?a Duri? >> > From dragisha at m3w.org Mon Apr 23 10:21:16 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 23 Apr 2007 10:21:16 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> Message-ID: <1177316476.737.97.camel@faramir.m3w.org> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > > > Hello, > > > > Have been skimming through source of PTHREAD code... And I think > > job can > > be done without so much relying on how-they-did-it-before, esp with > > regard to list of waiters and similar internal and global structures. > > Also, I see number of global locks and I am sure they are congestion > > generators every now and while, esp in heavy threading situations. > > > > Of course, there is number of approaches to this multi-thread > > situations. Mine being one of very nonconservative use of threads, I > > think it is important to remain open to possibly very big number of > > threads running in single process - meaning scalability is one of > > primary objections... As global locks don't do well with > > scalability, I > > don't like "cm" and similar global congestion points. > > Yes, there are tensions between a thin/absent veneer between language > threads and system threads. Most important are issues of preserving > a reasonable memory model for programmers (see Hans Boehm's paper > http://portal.acm.org/citation.cfm?doid=1065010.1065042). I know that paper, and being Modula-3 camp, I am - by definition - agreed to no-library-approach-for-threads :). > There are > also questions of portability and debuggability. Of course. That's why I am using only POSIX defined features, and when in doubt - ones used by Boehm in his famous GC :). > I agree that global > locks are to be avoided where they cause known contention, but there > are tradeoffs there too. Global lock is bad, whatever reasons. > For large numbers of threads (as you appear > to need) I think we would need to adopt some other implementation > approach, possibly by multiplexing multiple lightweight user-level > threads on some smaller number of heavyweight system-level threads, > but then you run into scheduling and load-balancing problems. I've argued this before... With O(1) process scheduling available for four years from Linux, and in that time surely from everyone else... It would be bigger problem to maintain scheduling for special "BIG" cases inside our support libraries than to rely on operating system. It's good that mainstream OS people recognized threads as Need, and it is time now for us to accept they did it well - AT LAST. And, very important... I can't see what is heavyweight on system which does 10,000 context switches per second for 1500 threads with 2% CPU load. And all this in 2004 on some 1.something GHz CPU. Threads WERE heavyweight four years ago, and they are not, long since. Even Windows has lightweight kernel-space threads :). > > > We talked about this at least once before, and I think I remember you > > insisted on more compatibility than can be read from SPwM3. Do you > > think > > best idea would be to integrate mine NPTL code into CM3 for people to > > trash and test, and let everyone select what is best for their > > situation? > > What I wanted was a situation where programs would be able to run > with the same tools (e.g., showheap, showthread) under both user- > level and system threading. This goal has been achieved with the > current pthread-based implementation. It is good reasoning, and it's one of reasons I did not suggest replacement... I think mine version is less bloated and I know it's very good for long and stable process uptime we all expect from Modula-3 programs. But also, implied compatibilities outside of SPwM3 and direct demands from other parts of runtime were not on my list. These I've respected, and it looks like these are good production time criteria. As opposed to excellent development time criteria you based yours on. > Moreover, I wanted something > where variations in thread support from one system to another could > be exploited most easily (such as for systems where thread suspend/ > resume is provided as a primitive). Again, the current > implementation achieves this, and runs with minimal target-specific > code on Darwin, Solaris, and Linux. Ports to other targets should be > relatively straightforward. I've not ported mine outside of LINUXLIBC6, but as it's extremmely POSIX, I see no problem. > > > Problems I had with my pthread implementation were all related to VM > > hell of earlier GC implementation... After you did that piece of art > > with new approach to GC, I expect infinite uptimes from my servers and > > bots :). Big thanks for that! > > Any native threading implementation is going to have problems with VM- > based memory management. I'm surprised to were able to get anything > going with the VM-based GC. Anything is pretty much - I have heavy multithreaded servers running literally for years,,, one of them is up since January of 2004, it services few hundreds of connected users (and up to 1500 threads) at almost every moment and breaks only when system reboots :). All that with heavy integration of various C libraries. > > > > > dd > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From hosking at cs.purdue.edu Mon Apr 23 14:40:44 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 23 Apr 2007 08:40:44 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177316476.737.97.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: I take all of your points seriously. One option would be to offer your threads implementation as another build option for CM3. I'm going to track down the bug I introduced recently and then we can consider how to move forward. On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >> >>> Hello, >>> >>> Have been skimming through source of PTHREAD code... And I think >>> job can >>> be done without so much relying on how-they-did-it-before, esp with >>> regard to list of waiters and similar internal and global >>> structures. >>> Also, I see number of global locks and I am sure they are congestion >>> generators every now and while, esp in heavy threading situations. >>> >>> Of course, there is number of approaches to this multi-thread >>> situations. Mine being one of very nonconservative use of threads, I >>> think it is important to remain open to possibly very big number of >>> threads running in single process - meaning scalability is one of >>> primary objections... As global locks don't do well with >>> scalability, I >>> don't like "cm" and similar global congestion points. >> >> Yes, there are tensions between a thin/absent veneer between language >> threads and system threads. Most important are issues of preserving >> a reasonable memory model for programmers (see Hans Boehm's paper >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > I know that paper, and being Modula-3 camp, I am - by definition - > agreed to no-library-approach-for-threads :). > >> There are >> also questions of portability and debuggability. > > Of course. That's why I am using only POSIX defined features, and > when > in doubt - ones used by Boehm in his famous GC :). > >> I agree that global >> locks are to be avoided where they cause known contention, but there >> are tradeoffs there too. > > Global lock is bad, whatever reasons. > >> For large numbers of threads (as you appear >> to need) I think we would need to adopt some other implementation >> approach, possibly by multiplexing multiple lightweight user-level >> threads on some smaller number of heavyweight system-level threads, >> but then you run into scheduling and load-balancing problems. > > I've argued this before... With O(1) process scheduling available > for > four years from Linux, and in that time surely from everyone > else... It > would be bigger problem to maintain scheduling for special "BIG" cases > inside our support libraries than to rely on operating system. It's > good > that mainstream OS people recognized threads as Need, and it is > time now > for us to accept they did it well - AT LAST. > > And, very important... I can't see what is heavyweight on system > which > does 10,000 context switches per second for 1500 threads with 2% CPU > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > heavyweight four years ago, and they are not, long since. Even Windows > has lightweight kernel-space threads :). > >> >>> We talked about this at least once before, and I think I remember >>> you >>> insisted on more compatibility than can be read from SPwM3. Do you >>> think >>> best idea would be to integrate mine NPTL code into CM3 for >>> people to >>> trash and test, and let everyone select what is best for their >>> situation? >> >> What I wanted was a situation where programs would be able to run >> with the same tools (e.g., showheap, showthread) under both user- >> level and system threading. This goal has been achieved with the >> current pthread-based implementation. > > It is good reasoning, and it's one of reasons I did not suggest > replacement... I think mine version is less bloated and I know it's > very > good for long and stable process uptime we all expect from Modula-3 > programs. But also, implied compatibilities outside of SPwM3 and > direct > demands from other parts of runtime were not on my list. These I've > respected, and it looks like these are good production time > criteria. As > opposed to excellent development time criteria you based yours on. > >> Moreover, I wanted something >> where variations in thread support from one system to another could >> be exploited most easily (such as for systems where thread suspend/ >> resume is provided as a primitive). Again, the current >> implementation achieves this, and runs with minimal target-specific >> code on Darwin, Solaris, and Linux. Ports to other targets should be >> relatively straightforward. > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > POSIX, I see no problem. > >> >>> Problems I had with my pthread implementation were all related to VM >>> hell of earlier GC implementation... After you did that piece of art >>> with new approach to GC, I expect infinite uptimes from my >>> servers and >>> bots :). Big thanks for that! >> >> Any native threading implementation is going to have problems with >> VM- >> based memory management. I'm surprised to were able to get anything >> going with the VM-based GC. > > Anything is pretty much - I have heavy multithreaded servers running > literally for years,,, one of them is up since January of 2004, it > services few hundreds of connected users (and up to 1500 threads) at > almost every moment and breaks only when system reboots :). All that > with heavy integration of various C libraries. > >> >>> >>> dd >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From mika at async.caltech.edu Tue Apr 24 22:21:25 2007 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 24 Apr 2007 13:21:25 -0700 Subject: [M3devel] order of module initialization... In-Reply-To: Your message of "Tue, 24 Apr 2007 14:15:19 -0000." Message-ID: <200704242021.l3OKLPac036051@camembert.async.caltech.edu> Yes, that is the entire rule. There can definitely be cycles in the graph; in this case, the rule "if M depends on N and N does not depend on M" does not apply, as they depend on each other. The order of initialization is then implementation-dependent. Just creating objects of type (circularly) implemented by another module doesn't present any special problems, unless the correct functioning of those objects is dependent on module-initialization code. And I don't know about your compiler, but mine won't link things that didn't compile. I have to admit that the module initialization rule does break a nice property of Modula-3 programs, which is that you can't normally break a module by making a change to another module. If your module M indirectly depends on module N and furthermore, the correct functioning of the M-N relationship depends on module N's being initialized before module M, the addition in module N (in private implementation code) of a dependency on module M's exported interface can cause your program to fail. Mika "j k" writes: >ok. Can there be circularities in the graph? >Two modules creating objects of types implemented by the other in their >module initializer? Or that won't compile? I'll have to try it. > >- Jay > > >>From: Mika Nystrom >>To: "j k" >>CC: dragisha at m3w.org, m3devel at elegosoft.com >>Subject: Re: [M3devel] order of module initialization... Date: Fri, 20 Apr >>2007 15:15:16 -0700 >> >>The order in Modula-3 is defined in section 2.5.6 of the language >>report (page 47 of Systems Programming with Modula-3). >> >>The problem described here appears to be that EXPORTS isn't treated >>the same as IMPORT (which it should be). There are some other >>problems I noticed when porting PM3 code to CM3, don't know if they >>have been fixed. The "solution" in that case was to IMPORT interfaces >>in places where they weren't really needed. >> >>Obfuscating one's code to avoid taking advantage of the convenience >>of module initialization seems like a rather severe suggestion... >> >>The rule, by the way, is: >> >>"If module M depends on module N and N does not depend on M, then >>N's body will be executed before M's body, where: >> >>* A module M 'depends on' a module N if M uses an interface that N >>exports or if M depends on a module that depends on N. >> >>* A module M 'uses' an interface I if M imports or exports I or if >>M uses an interface that imports I. >> >>Except for this constraint, the order of execution is >>implementation-dependent." >> >> Mika >> >>"j k" writes: >> >Reminds me of C++ globals with constructors. >> > >> >The solution is to avoid initialization. >> > >> >Do everything "on demand" and hide all state behind functions. >> > and then "on demand" needs to be thread safe, so there is a small >> >bootstrapping problem -- basically the only initialization you should do, >> >speaking in Win32-ese, is a few calls to InitializeCriticalSection and >>maybe >> >TlsAlloc and maybe HeapAlloc, but nothing "cross module", or more >> >accurately, nothing "in the same layer", if you have a dependency graph, >> >only "to a lower layer", which InitializeCritical/TlsAlloc nearly always >>are >> >(unless you are working on ntdll.dll, but even then.. :) ) If you don't >>have >> >a strict layered design, or can't capture what it is and analyze >>statically, >> >again, just restricting yourself to InitializeCriticalSection/TlsAlloc is >>an >> >easy way to be correct. >> > >> >And even then, you can often avoid the critical sections via something >>like >> >InterlockedCompareExchangePointer, even implement spin locks for this >rare >> >code. (Hand written spin locks are generally to be avoided since the >>kernel >> >won't know about them and you won't schedule efficiently, whereas with >> >EnterCriticalSection, if there is contention, the kernel has a hint what >>is >> >going on and can schedule better). >> > >> >Don't have any "public global data", unless, and I don't know if this >> >applies in Modula-3, unless it is "constant initialized", which I'm not >>sure >> >is well defined in C++ or not. I do believe it is well defined in >>C..except >> >dynamic linking I think screws that up to. >> > >> >(pardon the Win32-centricity; I'm sure the principles apply more broadly) >> > >> >Ok, in C++ and in Modula-3, there are maybe better answers. >> >In C++, globals within a source file are guaranteed to be initialized in >>the >> >order they occur in the file. >> >But the order of files is not defined. >> >There is a documented trick..potentially very inefficient..that I don't >>have >> >the patience to describe here.. "the iostream counter trick"... this I >>don't >> >think has an analog in Modula-3, but sure. >> >Stroupstroup's Design and Evolution book documents it. >> > >> >- Jay >> > >> > >> >>From: Dragi??a Duri?? >> >>To: "M3 Developers' Mailing List" >> >>Subject: [M3devel] order of module initialization... >> >>Date: Fri, 20 Apr 2007 21:36:25 +0200 >> >> >> >>I remember debugging this first time we tried to migrate to CM3... And >> >>then we solved it, for current CM3 and current version of our software, >> >>mentioning some imported module in EXPORTS clause... at the moment, this >> >>linked these modules in initialization chain and it worked ok... Right >> >>now, it does not. >> >> >> >>Module XLBuiltin is one imported regularly, and XLModuleUDP (for >> >>example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to >> >>initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ >> >>wrong moment... >> >> >> >>What would be solution for this? >> >> >> >>dd >> >>-- >> >>Dragi??a Duri?? >> >> >> >>_______________________________________________ >> >>M3devel mailing list >> >>M3devel at elegosoft.com >> >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >_________________________________________________________________ >> >Download Messenger. Join the i?m Initiative. Help make a difference >>today. >> >http://im.live.com/messenger/im/home/?source=TAGHM_APR07 >> > >> >_______________________________________________ >> >M3devel mailing list >> >M3devel at elegosoft.com >> >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >_________________________________________________________________ >Mortgage rates near historic lows. Refinance $200,000 loan for as low as >$771/month* >https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&search=m >ortgage_text_links_88_h27f8&disc=y&vers=689&s=4056&p=5117 From dragisha at m3w.org Wed Apr 25 08:55:12 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 08:55:12 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> Message-ID: <1177484112.4826.9.camel@faramir.m3w.org> Just a random thought... I don't think my TestThreads is something special, but it's few thread use patterns combined... And I've just had bright :) idea yesterday - it's also decent benchmark for whole threading system... I think it would be nice to test it with 10000 rounds, 4000 threads each (once cm3 cvs-head is fixed) with PTHREAD and without PTHREAD. I will do tests for mine. I think these extra data structures and global locks in PTHREAD are big efficiency killers. Benchmark will show. dd On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > I take all of your points seriously. One option would be to offer > your threads implementation as another build option for CM3. I'm > going to track down the bug I introduced recently and then we can > consider how to move forward. > > On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > > > On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >> > >>> Hello, > >>> > >>> Have been skimming through source of PTHREAD code... And I think > >>> job can > >>> be done without so much relying on how-they-did-it-before, esp with > >>> regard to list of waiters and similar internal and global > >>> structures. > >>> Also, I see number of global locks and I am sure they are congestion > >>> generators every now and while, esp in heavy threading situations. > >>> > >>> Of course, there is number of approaches to this multi-thread > >>> situations. Mine being one of very nonconservative use of threads, I > >>> think it is important to remain open to possibly very big number of > >>> threads running in single process - meaning scalability is one of > >>> primary objections... As global locks don't do well with > >>> scalability, I > >>> don't like "cm" and similar global congestion points. > >> > >> Yes, there are tensions between a thin/absent veneer between language > >> threads and system threads. Most important are issues of preserving > >> a reasonable memory model for programmers (see Hans Boehm's paper > >> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > > > > I know that paper, and being Modula-3 camp, I am - by definition - > > agreed to no-library-approach-for-threads :). > > > >> There are > >> also questions of portability and debuggability. > > > > Of course. That's why I am using only POSIX defined features, and > > when > > in doubt - ones used by Boehm in his famous GC :). > > > >> I agree that global > >> locks are to be avoided where they cause known contention, but there > >> are tradeoffs there too. > > > > Global lock is bad, whatever reasons. > > > >> For large numbers of threads (as you appear > >> to need) I think we would need to adopt some other implementation > >> approach, possibly by multiplexing multiple lightweight user-level > >> threads on some smaller number of heavyweight system-level threads, > >> but then you run into scheduling and load-balancing problems. > > > > I've argued this before... With O(1) process scheduling available > > for > > four years from Linux, and in that time surely from everyone > > else... It > > would be bigger problem to maintain scheduling for special "BIG" cases > > inside our support libraries than to rely on operating system. It's > > good > > that mainstream OS people recognized threads as Need, and it is > > time now > > for us to accept they did it well - AT LAST. > > > > And, very important... I can't see what is heavyweight on system > > which > > does 10,000 context switches per second for 1500 threads with 2% CPU > > load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > > heavyweight four years ago, and they are not, long since. Even Windows > > has lightweight kernel-space threads :). > > > >> > >>> We talked about this at least once before, and I think I remember > >>> you > >>> insisted on more compatibility than can be read from SPwM3. Do you > >>> think > >>> best idea would be to integrate mine NPTL code into CM3 for > >>> people to > >>> trash and test, and let everyone select what is best for their > >>> situation? > >> > >> What I wanted was a situation where programs would be able to run > >> with the same tools (e.g., showheap, showthread) under both user- > >> level and system threading. This goal has been achieved with the > >> current pthread-based implementation. > > > > It is good reasoning, and it's one of reasons I did not suggest > > replacement... I think mine version is less bloated and I know it's > > very > > good for long and stable process uptime we all expect from Modula-3 > > programs. But also, implied compatibilities outside of SPwM3 and > > direct > > demands from other parts of runtime were not on my list. These I've > > respected, and it looks like these are good production time > > criteria. As > > opposed to excellent development time criteria you based yours on. > > > >> Moreover, I wanted something > >> where variations in thread support from one system to another could > >> be exploited most easily (such as for systems where thread suspend/ > >> resume is provided as a primitive). Again, the current > >> implementation achieves this, and runs with minimal target-specific > >> code on Darwin, Solaris, and Linux. Ports to other targets should be > >> relatively straightforward. > > > > I've not ported mine outside of LINUXLIBC6, but as it's extremmely > > POSIX, I see no problem. > > > >> > >>> Problems I had with my pthread implementation were all related to VM > >>> hell of earlier GC implementation... After you did that piece of art > >>> with new approach to GC, I expect infinite uptimes from my > >>> servers and > >>> bots :). Big thanks for that! > >> > >> Any native threading implementation is going to have problems with > >> VM- > >> based memory management. I'm surprised to were able to get anything > >> going with the VM-based GC. > > > > Anything is pretty much - I have heavy multithreaded servers running > > literally for years,,, one of them is up since January of 2004, it > > services few hundreds of connected users (and up to 1500 threads) at > > almost every moment and breaks only when system reboots :). All that > > with heavy integration of various C libraries. > > > >> > >>> > >>> dd > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri? From wagner at elegosoft.com Wed Apr 25 09:10:20 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:10:20 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <1177097786.4282.75.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> Message-ID: <462EFEDC.5090000@elegosoft.com> Dragi?a Duri? wrote: > I remember debugging this first time we tried to migrate to CM3... And > then we solved it, for current CM3 and current version of our software, > mentioning some imported module in EXPORTS clause... at the moment, this > linked these modules in initialization chain and it worked ok... Right > now, it does not. > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > wrong moment... > > What would be solution for this? > The order of module initialization in CM3 is extensively traced. Try to run your program with @M3tracelinker and see what's wrong. The code for module initialization is in cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 If the initialization does not work as expected, we must be missing of overlooking some dependency there. If you can provide a simple and small example that does not work, I can probably have a look at it at the next weekend. Sorry for the late answer, Olaf From dragisha at m3w.org Wed Apr 25 09:44:52 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 09:44:52 +0200 Subject: [M3devel] order of module initialization... In-Reply-To: <462EFEDC.5090000@elegosoft.com> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> Message-ID: <1177487092.4826.15.camel@faramir.m3w.org> I've debugged this before, and I know the drill... Problem with initialization, sometimes, is in it passing pass 2 of RTLinker for some modules and never coming to pass 3... I've noticed that. Last few days I am using -linkall, and after I've deleted some redundant EXPORTS it kind of works ok. I remember you telling me about this non-initialization of non-imported modules once before. It is where CM3 differs from both SRC and PM3 in big way. Don't remember rationalization, but I don't like situation where I can't make/exploit plugin architecture simply by mentioning new module in m3makefile and letting it initialize itself in standardized manner. dd On Wed, 2007-04-25 at 09:10 +0200, Olaf Wagner wrote: > Dragi?a Duri? wrote: > > I remember debugging this first time we tried to migrate to CM3... And > > then we solved it, for current CM3 and current version of our software, > > mentioning some imported module in EXPORTS clause... at the moment, this > > linked these modules in initialization chain and it worked ok... Right > > now, it does not. > > > > Module XLBuiltin is one imported regularly, and XLModuleUDP (for > > example) both EXPORTS and IMPORTs XLBuiltin. It is linked and starts to > > initialize, but _before_ XLBuiltin... And, fails. Something is NIL @ > > wrong moment... > > > > What would be solution for this? > > > The order of module initialization in CM3 is extensively traced. Try to > run your program with @M3tracelinker and see what's wrong. The > code for module initialization is in > > cm3/m3-libs/m3core/src/runtime/common/RTLinker.m3 > > If the initialization does not work as expected, we must be missing > of overlooking some dependency there. > > If you can provide a simple and small example that does not work, > I can probably have a look at it at the next weekend. > > Sorry for the late answer, > > Olaf -- Dragi?a Duri? From wagner at mx0.elegosoft.com Wed Apr 25 09:00:22 2007 From: wagner at mx0.elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 09:00:22 +0200 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> Message-ID: <20070425070022.GA4827@elegosoft.com> On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: > I have my hands full with GC evolution and thread issues at the > moment. Anyone else care to look into this? I've checked in a fix for this monday evening, thought the commit message again wasn't delivered. I also instructed Ronny to have a look at the commit message setup again. Olaf > On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: > > >while trying to make my programs with newsly installed cm3, I am > >constantly running into same error. Example: > > > >import("libm3") > > > >include_dir("../../../../m3lib0/src") > >include_dir("../../../../pdf/src") > >include_dir("../../../../zip/src") > > > >implementation("Test") > >program("test") > > > >Is my m3makefile, and when "cm3" is run in package's src dir, I have > >only: > > > >faramir:dragisha/pts/9: test/1/src% cm3 > >--- building in ../LINUXLIBC6 --- > > > > > > > >*** > >*** runtime error: > >*** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES > >list > >*** file "../src/os/POSIX/PathnamePosix.m3", line 98 > >*** > > > >Mentioned line raises this FATAL exception when Absolute(path) is > >TRUE... What is absolute in my paths? > > > >All m3makefiles in question were ok with pm3, of course.. > > > >dd > >-- > >Dragi??a Duri?? > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Wed Apr 25 11:33:44 2007 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 25 Apr 2007 11:33:44 +0200 (CEST) Subject: [M3devel] order of module initialization... In-Reply-To: <1177487092.4826.15.camel@faramir.m3w.org> References: <1177097786.4282.75.camel@faramir.m3w.org> <462EFEDC.5090000@elegosoft.com> <1177487092.4826.15.camel@faramir.m3w.org> Message-ID: <62295.194.138.127.36.1177493624.squirrel@mail.elegosoft.com> On Wed, April 25, 2007 9:44 am, Dragi??a Duri?� wrote: > I've debugged this before, and I know the drill... Problem with > initialization, sometimes, is in it passing pass 2 of RTLinker for some > modules and never coming to pass 3... I've noticed that. Last few days I > am using -linkall, and after I've deleted some redundant EXPORTS it kind > of works ok. > > I remember you telling me about this non-initialization of non-imported > modules once before. It is where CM3 differs from both SRC and PM3 in > big way. Don't remember rationalization, but I don't like situation > where I can't make/exploit plugin architecture simply by mentioning new > module in m3makefile and letting it initialize itself in standardized > manner. I'm afraid I still don't understand the problem well enough though I think we've probably got a bug somewhere in the initialization code. It would be really helpful if you could provide a simple example for debugging. I'll take care of the problem then (though it may take some time). Olaf -- Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Apr 25 15:51:52 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:51:52 -0400 Subject: [M3devel] pthread in cm3... In-Reply-To: <1177484112.4826.9.camel@faramir.m3w.org> References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: Yes, good thinking. Tuning the threads systems is a good plan all round. On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > Just a random thought... > > I don't think my TestThreads is something special, but it's few thread > use patterns combined... And I've just had bright :) idea yesterday - > it's also decent benchmark for whole threading system... I think it > would be nice to test it with 10000 rounds, 4000 threads each (once > cm3 > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > tests for > mine. > > I think these extra data structures and global locks in PTHREAD are > big > efficiency killers. Benchmark will show. > > dd > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: >> I take all of your points seriously. One option would be to offer >> your threads implementation as another build option for CM3. I'm >> going to track down the bug I introduced recently and then we can >> consider how to move forward. >> >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: >> >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: >>>> >>>>> Hello, >>>>> >>>>> Have been skimming through source of PTHREAD code... And I think >>>>> job can >>>>> be done without so much relying on how-they-did-it-before, esp >>>>> with >>>>> regard to list of waiters and similar internal and global >>>>> structures. >>>>> Also, I see number of global locks and I am sure they are >>>>> congestion >>>>> generators every now and while, esp in heavy threading situations. >>>>> >>>>> Of course, there is number of approaches to this multi-thread >>>>> situations. Mine being one of very nonconservative use of >>>>> threads, I >>>>> think it is important to remain open to possibly very big >>>>> number of >>>>> threads running in single process - meaning scalability is one of >>>>> primary objections... As global locks don't do well with >>>>> scalability, I >>>>> don't like "cm" and similar global congestion points. >>>> >>>> Yes, there are tensions between a thin/absent veneer between >>>> language >>>> threads and system threads. Most important are issues of >>>> preserving >>>> a reasonable memory model for programmers (see Hans Boehm's paper >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). >>> >>> I know that paper, and being Modula-3 camp, I am - by definition - >>> agreed to no-library-approach-for-threads :). >>> >>>> There are >>>> also questions of portability and debuggability. >>> >>> Of course. That's why I am using only POSIX defined features, and >>> when >>> in doubt - ones used by Boehm in his famous GC :). >>> >>>> I agree that global >>>> locks are to be avoided where they cause known contention, but >>>> there >>>> are tradeoffs there too. >>> >>> Global lock is bad, whatever reasons. >>> >>>> For large numbers of threads (as you appear >>>> to need) I think we would need to adopt some other implementation >>>> approach, possibly by multiplexing multiple lightweight user-level >>>> threads on some smaller number of heavyweight system-level threads, >>>> but then you run into scheduling and load-balancing problems. >>> >>> I've argued this before... With O(1) process scheduling available >>> for >>> four years from Linux, and in that time surely from everyone >>> else... It >>> would be bigger problem to maintain scheduling for special "BIG" >>> cases >>> inside our support libraries than to rely on operating system. It's >>> good >>> that mainstream OS people recognized threads as Need, and it is >>> time now >>> for us to accept they did it well - AT LAST. >>> >>> And, very important... I can't see what is heavyweight on system >>> which >>> does 10,000 context switches per second for 1500 threads with 2% CPU >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE >>> heavyweight four years ago, and they are not, long since. Even >>> Windows >>> has lightweight kernel-space threads :). >>> >>>> >>>>> We talked about this at least once before, and I think I remember >>>>> you >>>>> insisted on more compatibility than can be read from SPwM3. Do you >>>>> think >>>>> best idea would be to integrate mine NPTL code into CM3 for >>>>> people to >>>>> trash and test, and let everyone select what is best for their >>>>> situation? >>>> >>>> What I wanted was a situation where programs would be able to run >>>> with the same tools (e.g., showheap, showthread) under both user- >>>> level and system threading. This goal has been achieved with the >>>> current pthread-based implementation. >>> >>> It is good reasoning, and it's one of reasons I did not suggest >>> replacement... I think mine version is less bloated and I know it's >>> very >>> good for long and stable process uptime we all expect from Modula-3 >>> programs. But also, implied compatibilities outside of SPwM3 and >>> direct >>> demands from other parts of runtime were not on my list. These I've >>> respected, and it looks like these are good production time >>> criteria. As >>> opposed to excellent development time criteria you based yours on. >>> >>>> Moreover, I wanted something >>>> where variations in thread support from one system to another could >>>> be exploited most easily (such as for systems where thread suspend/ >>>> resume is provided as a primitive). Again, the current >>>> implementation achieves this, and runs with minimal target-specific >>>> code on Darwin, Solaris, and Linux. Ports to other targets >>>> should be >>>> relatively straightforward. >>> >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely >>> POSIX, I see no problem. >>> >>>> >>>>> Problems I had with my pthread implementation were all related >>>>> to VM >>>>> hell of earlier GC implementation... After you did that piece >>>>> of art >>>>> with new approach to GC, I expect infinite uptimes from my >>>>> servers and >>>>> bots :). Big thanks for that! >>>> >>>> Any native threading implementation is going to have problems with >>>> VM- >>>> based memory management. I'm surprised to were able to get >>>> anything >>>> going with the VM-based GC. >>> >>> Anything is pretty much - I have heavy multithreaded servers >>> running >>> literally for years,,, one of them is up since January of 2004, it >>> services few hundreds of connected users (and up to 1500 threads) at >>> almost every moment and breaks only when system reboots :). All that >>> with heavy integration of various C libraries. >>> >>>> >>>>> >>>>> dd >>>>> -- >>>>> Dragi?a Duri? >>>> >>> -- >>> Dragi?a Duri? >> > -- > Dragi?a Duri? From hosking at cs.purdue.edu Wed Apr 25 15:52:49 2007 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 25 Apr 2007 09:52:49 -0400 Subject: [M3devel] blatant :) incompatibility in m3makefiles In-Reply-To: <20070425070022.GA4827@elegosoft.com> References: <1177056440.4282.34.camel@faramir.m3w.org> <6FC282AA-3A4D-4F18-BD82-2C14203A7ED5@cs.purdue.edu> <20070425070022.GA4827@elegosoft.com> Message-ID: PS CVS head is broken right now due to my most recent "fixes" to threading. I am currently looking into the problem. On Apr 25, 2007, at 3:00 AM, Olaf Wagner wrote: > On Fri, Apr 20, 2007 at 09:16:33AM -0400, Tony Hosking wrote: >> I have my hands full with GC evolution and thread issues at the >> moment. Anyone else care to look into this? > > I've checked in a fix for this monday evening, thought the commit > message again wasn't delivered. I also instructed Ronny to have > a look at the commit message setup again. > > Olaf > >> On Apr 20, 2007, at 4:07 AM, Dragi??a Duri?? wrote: >> >>> while trying to make my programs with newsly installed cm3, I am >>> constantly running into same error. Example: >>> >>> import("libm3") >>> >>> include_dir("../../../../m3lib0/src") >>> include_dir("../../../../pdf/src") >>> include_dir("../../../../zip/src") >>> >>> implementation("Test") >>> program("test") >>> >>> Is my m3makefile, and when "cm3" is run in package's src dir, I have >>> only: >>> >>> faramir:dragisha/pts/9: test/1/src% cm3 >>> --- building in ../LINUXLIBC6 --- >>> >>> >>> >>> *** >>> *** runtime error: >>> *** Exception "PathnamePosix.CheckedRuntimeError" not in RAISES >>> list >>> *** file "../src/os/POSIX/PathnamePosix.m3", line 98 >>> *** >>> >>> Mentioned line raises this FATAL exception when Absolute(path) is >>> TRUE... What is absolute in my paths? >>> >>> All m3makefiles in question were ok with pm3, of course.. >>> >>> dd >>> -- >>> Dragi??a Duri?? >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Olaf Wagner -- elego Software Solutions GmbH, Ohmstr. 9, 10179 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 23 45 869 fax: +49 30 > 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | > Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 From dragisha at m3w.org Wed Apr 25 18:35:37 2007 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Wed, 25 Apr 2007 18:35:37 +0200 Subject: [M3devel] pthread in cm3... In-Reply-To: References: <1177253233.737.18.camel@faramir.m3w.org> <4B72258F-EB21-48A3-854F-2AB1A5C059F9@cs.purdue.edu> <1177316476.737.97.camel@faramir.m3w.org> <1177484112.4826.9.camel@faramir.m3w.org> Message-ID: <1177518937.15060.6.camel@faramir.m3w.org> After implementing that workaround for result code 11 in that SIGSUSPEND loop, every time during 1st or at most second pass (of 4000) it stucks. Not same place every time, though... I think there are RCs in LockHeap/SuspendAll logic. dd On Wed, 2007-04-25 at 09:51 -0400, Tony Hosking wrote: > Yes, good thinking. Tuning the threads systems is a good plan all > round. > > On Apr 25, 2007, at 2:55 AM, Dragi?a Duri? wrote: > > > Just a random thought... > > > > I don't think my TestThreads is something special, but it's few thread > > use patterns combined... And I've just had bright :) idea yesterday - > > it's also decent benchmark for whole threading system... I think it > > would be nice to test it with 10000 rounds, 4000 threads each (once > > cm3 > > cvs-head is fixed) with PTHREAD and without PTHREAD. I will do > > tests for > > mine. > > > > I think these extra data structures and global locks in PTHREAD are > > big > > efficiency killers. Benchmark will show. > > > > dd > > > > On Mon, 2007-04-23 at 08:40 -0400, Tony Hosking wrote: > >> I take all of your points seriously. One option would be to offer > >> your threads implementation as another build option for CM3. I'm > >> going to track down the bug I introduced recently and then we can > >> consider how to move forward. > >> > >> On Apr 23, 2007, at 4:21 AM, Dragi?a Duri? wrote: > >> > >>> On Sun, 2007-04-22 at 15:59 -0400, Tony Hosking wrote: > >>>> On Apr 22, 2007, at 10:47 AM, Dragi?a Duri? wrote: > >>>> > >>>>> Hello, > >>>>> > >>>>> Have been skimming through source of PTHREAD code... And I think > >>>>> job can > >>>>> be done without so much relying on how-they-did-it-before, esp > >>>>> with > >>>>> regard to list of waiters and similar internal and global > >>>>> structures. > >>>>> Also, I see number of global locks and I am sure they are > >>>>> congestion > >>>>> generators every now and while, esp in heavy threading situations. > >>>>> > >>>>> Of course, there is number of approaches to this multi-thread > >>>>> situations. Mine being one of very nonconservative use of > >>>>> threads, I > >>>>> think it is important to remain open to possibly very big > >>>>> number of > >>>>> threads running in single process - meaning scalability is one of > >>>>> primary objections... As global locks don't do well with > >>>>> scalability, I > >>>>> don't like "cm" and similar global congestion points. > >>>> > >>>> Yes, there are tensions between a thin/absent veneer between > >>>> language > >>>> threads and system threads. Most important are issues of > >>>> preserving > >>>> a reasonable memory model for programmers (see Hans Boehm's paper > >>>> http://portal.acm.org/citation.cfm?doid=1065010.1065042). > >>> > >>> I know that paper, and being Modula-3 camp, I am - by definition - > >>> agreed to no-library-approach-for-threads :). > >>> > >>>> There are > >>>> also questions of portability and debuggability. > >>> > >>> Of course. That's why I am using only POSIX defined features, and > >>> when > >>> in doubt - ones used by Boehm in his famous GC :). > >>> > >>>> I agree that global > >>>> locks are to be avoided where they cause known contention, but > >>>> there > >>>> are tradeoffs there too. > >>> > >>> Global lock is bad, whatever reasons. > >>> > >>>> For large numbers of threads (as you appear > >>>> to need) I think we would need to adopt some other implementation > >>>> approach, possibly by multiplexing multiple lightweight user-level > >>>> threads on some smaller number of heavyweight system-level threads, > >>>> but then you run into scheduling and load-balancing problems. > >>> > >>> I've argued this before... With O(1) process scheduling available > >>> for > >>> four years from Linux, and in that time surely from everyone > >>> else... It > >>> would be bigger problem to maintain scheduling for special "BIG" > >>> cases > >>> inside our support libraries than to rely on operating system. It's > >>> good > >>> that mainstream OS people recognized threads as Need, and it is > >>> time now > >>> for us to accept they did it well - AT LAST. > >>> > >>> And, very important... I can't see what is heavyweight on system > >>> which > >>> does 10,000 context switches per second for 1500 threads with 2% CPU > >>> load. And all this in 2004 on some 1.something GHz CPU. Threads WERE > >>> heavyweight four years ago, and they are not, long since. Even > >>> Windows > >>> has lightweight kernel-space threads :). > >>> > >>>> > >>>>> We talked about this at least once before, and I think I remember > >>>>> you > >>>>> insisted on more compatibility than can be read from SPwM3. Do you > >>>>> think > >>>>> best idea would be to integrate mine NPTL code into CM3 for > >>>>> people to > >>>>> trash and test, and let everyone select what is best for their > >>>>> situation? > >>>> > >>>> What I wanted was a situation where programs would be able to run > >>>> with the same tools (e.g., showheap, showthread) under both user- > >>>> level and system threading. This goal has been achieved with the > >>>> current pthread-based implementation. > >>> > >>> It is good reasoning, and it's one of reasons I did not suggest > >>> replacement... I think mine version is less bloated and I know it's > >>> very > >>> good for long and stable process uptime we all expect from Modula-3 > >>> programs. But also, implied compatibilities outside of SPwM3 and > >>> direct > >>> demands from other parts of runtime were not on my list. These I've > >>> respected, and it looks like these are good production time > >>> criteria. As > >>> opposed to excellent development time criteria you based yours on. > >>> > >>>> Moreover, I wanted something > >>>> where variations in thread support from one system to another could > >>>> be exploited most easily (such as for systems where thread suspend/ > >>>> resume is provided as a primitive). Again, the current > >>>> implementation achieves this, and runs with minimal target-specific > >>>> code on Darwin, Solaris, and Linux. Ports to other targets > >>>> should be > >>>> relatively straightforward. > >>> > >>> I've not ported mine outside of LINUXLIBC6, but as it's extremmely > >>> POSIX, I see no problem. > >>> > >>>> > >>>>> Problems I had with my pthread implementation were all related > >>>>> to VM > >>>>> hell of earlier GC implementation... After you did that piece > >>>>> of art > >>>>> with new approach to GC, I expect infinite uptimes from my > >>>>> servers and > >>>>> bots :). Big thanks for that! > >>>> > >>>> Any native threading implementation is going to have problems with > >>>> VM- > >>>> based memory management. I'm surprised to were able to get > >>>> anything > >>>> going with the VM-based GC. > >>> > >>> Anything is pretty much - I have heavy multithreaded servers > >>> running > >>> literally for years,,, one of them is up since January of 2004, it > >>> services few hundreds of connected users (and up to 1500 threads) at > >>> almost every moment and breaks only when system reboots :). All that > >>> with heavy integration of various C libraries. > >>> > >>>> > >>>>> > >>>>> dd > >>>>> -- > >>>>> Dragi?a Duri? > >>>> > >>> -- > >>> Dragi?a Duri? > >> > > -- > > Dragi?a Duri? > -- Dragi?a Duri?