From jay.krell at cornell.edu Thu Aug 2 09:28:40 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Message-ID: Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 2 09:38:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:38:52 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: References: Message-ID: okok..4.5 generates: ldx [%fp+2039], %g1 and %g1, 7, %g1 and %g1, 0xff, %g1 cmp %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: andcc %i0, 7, %g0 bne,pn %xcc, .LL2 - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 2 17:36:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 2 Aug 2012 16:36:40 +0100 (BST) Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: Message-ID: <1343921800.38422.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: but what pointers are you trying to diagnose, if it's a function closure then a REF to a RECORD paired with PROCEDURE is, that is, an OBJECT, but function "tagged" types, I don't think are M3language elements, that doesn't make them BRANDED by default, but structural equivalent by default just different for each type of procedure or method, if you think so, many languages have them, included C I guess, but why do we need those to build the compiler itself, perhaps object oriented environments can look for those in question, you can look for a hack B Kallsow tried to do in m3browser with RTType but I guess for structural types not branded types, so perhaps we need them both, it could be a nice work to do in static time, DEC experimented with some HW as a target for Cardelli's typeful Quest language, we could ask more about that experiment, since it targeted Modules static info and we could add everything you want in that. I need more time to do so, but looks interesting to do. Thanks in advance --- El jue, 2/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] unaligned procedures closure marker check broken? Para: "m3devel" Fecha: jueves, 2 de agosto, 2012 02:38 okok..4.5 generates: ??? ldx??? [%fp+2039], %g1 ??? and??? %g1, 7, %g1 ??? and??? %g1, 0xff, %g1 ??? cmp??? %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: ??? andcc??? %i0, 7, %g0 ??? bne,pn??? %xcc, .LL2 ?- Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89)???????? start_call_indirect type:addr (90)?????????? load_integer type:int_64 0 (91)?????????? pop_param type:int_64 (92)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93)?????????? loophole type1:addr type2:int_64 (94)?????????? load_integer type:int_64 7 (95)?????????? and type:int_64 (96)?????????? if_true type:int_64 label:1 frequency:0X32(50) (97)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98)?????????? load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: ?? ?save?? ?%sp, -208, %sp .LLCFI0: ?? ?stx?? ?%i0, [%fp+2175] ?? ?ldx?? ?[%fp+2175], %g1 ?? ?stx?? ?%g1, [%fp+2039] ?? ?ldx?? ?[%fp+2039], %g1 ?? ?and?? ?%g1, 7, %g1 <= correct ?? ?and?? ?%g1, 1, %g1? <= incorrect ?? ?and?? ?%g1, 0xff, %g1? <=? pointless, inefficient ?? ?cmp?? ?%g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { ? tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); ? EXPR_POP (); ? add_stmt (build3 (COND_EXPR, t_void, cond, ??????????????????? build1 (GOTO_EXPR, t_void, label), ??????????????????? NULL_TREE)); } static tree m3_cast (tree type, tree op0) { ? return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val;? true, false: Label;? freq: Frequency) = ? VAR skip := Next_label ();? nope := skip; ? BEGIN ??? IF (false # No_label) THEN? nope := false; END; ??? IF NOT Target.Aligned_procedures THEN ????? Push (proc); ????? Force (); ????? cg.loophole (Type.Addr, Target.Integer.cg_type); ????? Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); ????? cg.and (Target.Integer.cg_type); ???? ? ????? == insert comparisong to 0 here == ????? cg.if_true (Target.Integer.cg_type, nope, Always - freq); ????? SPop (1, "If_closure-unaligned"); ??? END; ?? ? and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 6 05:48:22 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 03:48:22 +0000 Subject: [M3devel] addresses of constants? Message-ID: Ugzip.m3 does an unnecessary heap allocation and copy. Here I have changed it to avoid the heap allocation. But it still makes a copy. How can I fix that? It seems ADR cannot be applied to constants? CONST ZLIB_VERSION = ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; This works but still is inefficient: PROCEDURE deflateInit(strm: z_stream_star; level: int): int = VAR version := ZLIB_VERSION; BEGIN RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); END deflateInit; I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 6 19:40:21 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 06 Aug 2012 12:40:21 -0500 Subject: [M3devel] addresses of constants? In-Reply-To: References: Message-ID: <50200185.1060106@lcwb.coop> You probably already know this, but, from 2.7: ADR (VAR x: Any) : ADDRESS ... ADR(x) is the address of the variable x. The actual argument must be a designator but need not be writable. And from 2.6.3: An identifier is a writable designator if it is declared as a variable, is a VAR or VALUE parameter, is a local of a TYPECASE or TRY-EXCEPT statement, or is a WITH local that is bound to a writable designator. An identifier is a readonly designator if it is a READONLY parameter, a local of a FOR statement, or a WITH local bound to a nondesignator or readonly designator. I'm speculating, but confidently, that the reason for the designator restriction is to have an implementation-independent rule that ensures the value would be in memory, and thus have an address, regardless of whether ADR is applied to it or not, The ADR could happen in a separate compilation from the declaration, and its existence be unknown at least until the main program was compiled. This way, compiler writers have the option of putting any constant in immediate instruction fields, if the target machine has them, without risk of undermining source code that compiled and ran fine on a different machine or with a different compiler. Even if it's sure to be in memory, allowing ADR(SomeConstant) also would open up the possibility of a constant that wasn't constant, a la FORTRAN. Yeah, I know it's already unsafe code anyway, but that would be an additional form of unsafety with scarcely an imaginable benefit. I'd just make it a VAR and put it inside a module. You would only have to verify that that one module didn't change it. It's possible a smart enough compiler would initialize right in the variable in the executable and not have to copy it. BTW, VAR ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) appears to have another problem. It assumes the representation of "1.0.4" is the same as, or at least contains ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; which it also looks like your modified UgzipP.deflateInit would expect. This happens to be true in pm3, but not cm3. Without debating the merits of various representations of TEXT literals, it would make for implementation-dependent code in any case. On 08/05/2012 10:48 PM, Jay K wrote: > Ugzip.m3 does an unnecessary heap allocation and copy. > > > Here I have changed it to avoid the heap allocation. > But it still makes a copy. > > How can I fix that? > > It seems ADR cannot be applied to constants? > > CONST > ZLIB_VERSION = ARRAY [0..5] OF char{ > ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; > > This works but still is inefficient: > > PROCEDURE deflateInit(strm: z_stream_star; level: int): int = > VAR version := ZLIB_VERSION; > BEGIN > RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); > END deflateInit; > > > I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); > const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. > > > - Jay From jay.krell at cornell.edu Tue Aug 7 01:02:16 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 23:02:16 +0000 Subject: [M3devel] initializer problem? In-Reply-To: References: <20120806105804.20F9F2474003@birch.elegosoft.com>, , , Message-ID: I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: DepthArray = REF ARRAY OF RECORD n := 0 END; dcount := NEW(DepthArray, 10); FOR i := FIRST(dcount^) TO LAST(dcount^) DO <* ASSERT dcount[i].n = 0 *> END; the assertion fails sometimes. I'll have to keep digging. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3 17 Mar 2008 03:46:14 -0000 1.2 +++ VBTRep.m3 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ (* All d: dcount[d].n = # VBTs in list with depth at most d. *) WITH v = NEW(REF ARRAY OF VBT.T, n) DO FOR i := 0 TO n - 1 DO + IF list[i].depth > 0 THEN v[dcount[list[i].depth - 1].n] := list[i].v; INC(dcount[list[i].depth - 1].n); (* All d: dcount[d-1].n = # VBTs in list with depth < d, or with depth = d that have been copied into v. *) END; + END; (* v has all the VBTs in the list and is sorted by depth *) FOR i := 0 TO n - 1 DO IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Tue Aug 7 15:44:27 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:44:27 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211BBB.4070203@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From mfleischer at elego.de Tue Aug 7 15:51:25 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:51:25 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211D5D.2040901@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Tue Aug 7 16:28:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 15:28:27 +0100 (BST) Subject: [M3devel] [cm3 trac maintenance notification] In-Reply-To: <50211BBB.4070203@elego.de> Message-ID: <1344349707.21021.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi: how much time, will be down, or just instantly be migrated? Well my hope is all bugs were gone but ... Thanks in advance --- El mar, 7/8/12, Maurizio Fleischer escribi?: De: Maurizio Fleischer Asunto: [M3devel] [cm3 trac maintenance notification] Para: m3devel at elego.de, m3announce at elego.de CC: admin at elego.de Fecha: martes, 7 de agosto, 2012 08:44 Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services&? Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96? ? ? maurizio.fleischer at elegosoft.com fax???+49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 7 19:50:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 18:50:31 +0100 (BST) Subject: [M3devel] initializer problem? In-Reply-To: Message-ID: <1344361831.31130.YahooMailClassic@web29701.mail.ird.yahoo.com> Hi all: interestingly, Greg Nelson mentioned here: http://books.google.com.co/books?id=EBW2AAAAIAAJ&q=trestle+%22dec-SRC%22&dq=trestle+%22dec-SRC%22&source=bl&ots=RTsXmdey5q&sig=XENEwmkvSMW3BzyeN9w_68VdcHU&hl=en&redir_esc=y that Trestle had some problem in RISC architectures, I wonder what he was referring to, and if they were working in a new design what did they changed. Thanks in advance --- El lun, 6/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] initializer problem? Para: "m3devel" Fecha: lunes, 6 de agosto, 2012 18:02 I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: ? DepthArray = REF ARRAY OF RECORD n := 0 END; ????? dcount := NEW(DepthArray, 10); ????? FOR i := FIRST(dcount^) TO LAST(dcount^) DO ??????? <* ASSERT dcount[i].n = 0 *> ????? END; the assertion fails sometimes. I'll have to keep digging. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3??? 17 Mar 2008 03:46:14 -0000??? 1.2 +++ VBTRep.m3??? 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ ?????? (* All d: dcount[d].n = # VBTs in list with depth at most d. *) ?????? WITH v = NEW(REF ARRAY OF VBT.T, n) DO ???????? FOR i := 0 TO n - 1 DO +????????? IF list[i].depth > 0 THEN ?????????? v[dcount[list[i].depth - 1].n] := list[i].v; ?????????? INC(dcount[list[i].depth - 1].n); ?????????? (* All d: dcount[d-1].n = # VBTs in list with depth < d, ????????????? or with depth = d that have been copied into v. *) ???????? END; +??????? END; ???????? (* v has all the VBTs in the list and is sorted by depth *) ???????? FOR i := 0 TO n - 1 DO ?????????? IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Wed Aug 8 16:00:14 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Wed, 08 Aug 2012 10:00:14 -0400 Subject: [M3devel] [cm3 trac maintenance notification] project back online In-Reply-To: <50211D5D.2040901@elego.de> References: <50211D5D.2040901@elego.de> Message-ID: <502270EE.80404@elego.de> Hi, the cm3 trac project is online again. As mentioned it is now reachable at https://cm3-bugs.elegosoft.com/cm3/. links in old ticket mails are redirected. kind regards elego systems administration On 08/07/2012 09:51 AM, Maurizio Fleischer wrote: > Hello, > due to maintenance work the cm3 trac project will be down tomorrow > 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is > reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users > accessing > https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make > your important changes beforehand and log out. > We apologize for the inconvenience. > > Thanks > elego systems administration > -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Sat Aug 11 16:20:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:20:40 +0100 (BST) Subject: [M3devel] Elegosoft.com mailer down? Message-ID: <1344694840.84378.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: How may times in a row could I send an malformed email? May I miss something m3devel ? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Aug 11 16:14:41 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:14:41 +0100 (BST) Subject: [M3devel] About extended char types, and integer types Message-ID: <1344694481.85507.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: I'm getting serious about this so I hope you don't misunderstand me, see this two differences in point of view of Modula-3 char character type and integer type systems: http://rant.gulbrandsen.priv.no/code/modula3-integers http://slps.svn.sourceforge.net/viewvc/slps/topics/grammars/modula/src-052/src.syntax.txt?view=markup&pathrev=1078 Though in some form, Modula-3 is better than this kind of things today are written,? if somebody takes the time to talk about it, it makes me think that something went wrong with today's software, perhaps that software is already dead. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 15 05:04:50 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 03:04:50 +0000 Subject: [M3devel] reference to globals in globals? Message-ID: Isn't it safe to take the address of a global? I have something like this: CONST UID_INTEGER = 1234; CONST UID_FLOAT = 4567; ... several more ... TYPE CType = OBJECT .. END; VAR t_int: CType := ...; VAR t_float: CType := ...; ... several more ... MapTypeIdToType(UID_INTEGER, t_int); MapTypeIdToType(UID_FLOAT, FLOAT); ... several more ... but what I really want is more like: TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: REF CType; END; CONST BuiltinUids = ARRAY OF BuiltinUids { BuiltinUids{UID_INTEGER, &t_int}, BuiltinUids{UID_FLOAT, &t_float}, ... several more ... }; FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); END; Heck, even if these weren't global, is it that unreasonble, from the programmer's point of view, for the language/compiler to do some pointer escape analysis and let me take the address of a local, as long as I don't store it somewhere that outlives the local? You can see this particular pattern currently in m3-sys/m3cc/gcc/gcc/m3cg/parse.c and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 where I encounter this. Working in safe languages can be frustrating... Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 15 16:51:41 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:51:41 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: Message-ID: <502BB77D.1030400@lcwb.coop> On 08/14/2012 10:04 PM, Jay K wrote: > Isn't it safe to take the address of a global? > Do you mean can't you use the ADR function in safe code if you apply it only to a global variable? The answer to that is no. The ADR function is illegal altogether in safe code. As to why, I can only speculate, but see below. I suspect even in this case, it is not as simple as it seems. > > I have something like this: > > > CONST UID_INTEGER = 1234; > CONST UID_FLOAT = 4567; > ... several more ... > > > TYPE CType = OBJECT .. END; > > > VAR t_int: CType := ...; > VAR t_float: CType := ...; > ... several more ... > > > MapTypeIdToType(UID_INTEGER, t_int); > MapTypeIdToType(UID_FLOAT, FLOAT); > ... several more ... > > > but what I really want is more like: > > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: REF CType; ^UNTRACED REF? If it were just REF, that would imply that your global variable (the pointer it contains) is a heap object, that it has heap allocator/GC overhead data attached to it, and that the GC should trace it, none of which is true. > END; > > > CONST BuiltinUids = ARRAY OF BuiltinUids { > BuiltinUids{UID_INTEGER, &t_int}, > BuiltinUids{UID_FLOAT, &t_float}, ADR instead of &? If so, you are still not there, because ADR returns a value of type ADDRESS, i.e., an untraced reference to we-don't-know-what. Somewhere, you would also have to use a LOOPHOLE to get it to UNTRACED REF CType. > ... several more ... > }; > > > FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > END; > I don't know what the signature of MapTypeIdToType is, but above, you are passing a variable of object type to its 2nd parameter, (which contains a traced reference to the actual heap object). But here, you pass the _address_ of the above. Inconsistent number of levels of indirection. A static safe language is much more likely to help with things like this. Maybe you just want to say TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: CType; and BuiltinUids{UID_INTEGER, t_int}? This would be equivalent to your first way, and doesn't require any unsafe coding at all. Or, you could do away with global variable t_int altogether and just initialize directly into BuiltinUids[..].ctype with whatever expression you used to initialize t_int. It looks like your array makes the t_int and cousins redundant. > > Heck, even if these weren't global, is it that unreasonble, > from the programmer's point of view, for the language/compiler > to do some pointer escape analysis and let me take the address > of a local, as long as I don't store it somewhere that outlives > the local? > This is ultimately an undecidable problem and even conservative approximations of reasonable sophistication are far too involved for a language to require of every compiler. > > You can see this particular pattern currently in > m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > > and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > where I encounter this. > > > Working in safe languages can be frustrating... > It's just an instant/deferred gratification thing. Safe languages often make you stop and fix it before you run it. Unsafe languages let you naively forge ahead to the next step, where the bug is likely to be *much* harder to diagnose, assuming you even have enough test cases to notice it at all during development. Your code here is a good example. Of course, safe languages occasionally make you unnecessarily write a bit more code to do it the safe way. E.g., the famous fake pointer to the root of a linked list example. In my personal experience, these times are at least one order of magnitude less frequent than the times safe languages reduce great pain to minor pain, albeit sooner. If fact, if you are accustomed to thinking in type-safe terms, it is seldom any harder to code it safely in the first place. You're making it too difficult. > > Thank you, > - Jay From rodney_bates at lcwb.coop Wed Aug 15 16:57:49 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:57:49 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20120815043215.C8614CC85C@birch.elegosoft.com> References: <20120815043215.C8614CC85C@birch.elegosoft.com> Message-ID: <502BB8ED.7030108@lcwb.coop> On 08/15/2012 06:32 AM, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/15 06:32:15 > > Modified files: > ./: Make-lang.in > > Log message: > compile parse.c as C++ even if compiling the rest of gcc as C (or however > it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in > (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes > too much to compile as C++, plus I'm not likely to fight this battle with 4.7) Does this mean we will now have to have a C++ compiler for every target we want M3 on? > > Modified files: > cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 > > Log message: > flush some test code > > From hosking at cs.purdue.edu Wed Aug 15 18:11:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 12:11:57 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: Jay, Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. I don?t know enough about your use-case to understand what you are trying to do. On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: >> Isn't it safe to take the address of a global? >> > > Do you mean can't you use the ADR function in safe code > if you apply it only to a global variable? The answer > to that is no. The ADR function is illegal altogether in > safe code. > > As to why, I can only speculate, but see below. I suspect > even in this case, it is not as simple as it seems. > >> >> I have something like this: >> >> >> CONST UID_INTEGER = 1234; >> CONST UID_FLOAT = 4567; >> ... several more ... >> >> >> TYPE CType = OBJECT .. END; >> >> >> VAR t_int: CType := ...; >> VAR t_float: CType := ...; >> ... several more ... >> >> >> MapTypeIdToType(UID_INTEGER, t_int); >> MapTypeIdToType(UID_FLOAT, FLOAT); >> ... several more ... >> >> >> but what I really want is more like: >> >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: REF CType; > > ^UNTRACED REF? If it were just REF, that would imply that > your global variable (the pointer it contains) is a heap object, that > it has heap allocator/GC overhead data attached to it, and that the GC > should trace it, none of which is true. > > >> END; >> >> >> CONST BuiltinUids = ARRAY OF BuiltinUids { >> BuiltinUids{UID_INTEGER, &t_int}, >> BuiltinUids{UID_FLOAT, &t_float}, > > ADR instead of &? If so, you are still not there, because ADR > returns a value of type ADDRESS, i.e., an untraced reference to > we-don't-know-what. Somewhere, you would also have to use a > LOOPHOLE to get it to UNTRACED REF CType. > >> ... several more ... >> }; >> >> >> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >> END; >> > > I don't know what the signature of MapTypeIdToType is, but above, > you are passing a variable of object type to its 2nd parameter, > (which contains a traced reference to the actual heap object). > But here, you pass the _address_ of the above. Inconsistent > number of levels of indirection. A static safe language is > much more likely to help with things like this. > > Maybe you just want to say > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: CType; > > and > > BuiltinUids{UID_INTEGER, t_int}? > > This would be equivalent to your first way, and doesn't require any > unsafe coding at all. > > Or, you could do away with global variable t_int altogether and > just initialize directly into BuiltinUids[..].ctype with whatever > expression you used to initialize t_int. It looks like your array > makes the t_int and cousins redundant. > >> >> Heck, even if these weren't global, is it that unreasonble, >> from the programmer's point of view, for the language/compiler >> to do some pointer escape analysis and let me take the address >> of a local, as long as I don't store it somewhere that outlives >> the local? >> > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. > >> >> You can see this particular pattern currently in >> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >> >> >> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >> where I encounter this. >> >> >> Working in safe languages can be frustrating... >> > > It's just an instant/deferred gratification thing. Safe languages often > make you stop and fix it before you run it. Unsafe languages let you naively > forge ahead to the next step, where the bug is likely to be *much* harder to > diagnose, assuming you even have enough test cases to notice it at all during > development. Your code here is a good example. > > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. In my personal experience, these times are at least > one order of magnitude less frequent than the times safe languages reduce > great pain to minor pain, albeit sooner. If fact, if you are accustomed to > thinking in type-safe terms, it is seldom any harder to code it safely in the > first place. > > You're making it too difficult. > >> >> Thank you, >> - Jay > From jay.krell at cornell.edu Wed Aug 15 18:56:09 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:56:09 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: <502BB77D.1030400@lcwb.coop> Message-ID: I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable? The answer >> to that is no. The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below. I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>> ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>> ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>> ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>> typeid: INTEGER; >>> ctype: REF CType; >> >> ^UNTRACED REF? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>> BuiltinUids{UID_INTEGER, &t_int}, >>> BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what. Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>> ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above. Inconsistent >> number of levels of indirection. A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int. It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing. Safe languages often >> make you stop and fix it before you run it. Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development. Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > From jay.krell at cornell.edu Wed Aug 15 18:50:04 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:50:04 -0400 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <502BB8ED.7030108@lcwb.coop> References: <20120815043215.C8614CC85C@birch.elegosoft.com> <502BB8ED.7030108@lcwb.coop> Message-ID: <34072368-7504-443C-B5EE-C0E75A75A68D@gmail.com> Yes that has already been the case for a long time. Gcc itself is also moving that way but isn't there yet. C++ compilers have long been widespread. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 10:57 AM, "Rodney M. Bates" wrote: > > > On 08/15/2012 06:32 AM, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 12/08/15 06:32:15 >> >> Modified files: >> ./: Make-lang.in >> >> Log message: >> compile parse.c as C++ even if compiling the rest of gcc as C (or however >> it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in >> (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes >> too much to compile as C++, plus I'm not likely to fight this battle with 4.7) > > Does this mean we will now have to have a C++ compiler for every target we want > M3 on? > >> >> Modified files: >> cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 >> >> Log message: >> flush some test code >> >> > From dabenavidesd at yahoo.es Wed Aug 15 21:46:39 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 15 Aug 2012 20:46:39 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345059808.37105.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <1345059999.77923.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: we need to calculate first Module instantiation in Modula-3 in SPIN MODULE_UNIT, that is trying to establish first Module instantiation, initialization, and then you can extract CONST "address" in the actual case is similar to that, a pointer to an address of the actual value, if pointer is NIL valued then it needed initialization in compile time else is a RT error to use its value, as we have proposed in other cases. Jay, please see this BitC language, has constructors for all those RT structures of your new back-end (hope you enjoy it, see p. 14, s. 3.6.2, Tag Representation): http://bitc-lang.org/docs/bitc/spec.pdf ). This leads in some special architectures to a double pointer-sized machine word type, which is why I talked about LONGADDRESS proposal. Thanks in advance --- El mi?, 15/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] reference to globals in globals? Para: "Tony Hosking" CC: "m3devel at elegosoft.com developers" , "Jay K" Fecha: mi?rcoles, 15 de agosto, 2012 11:56 I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array.? More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable?? The answer >> to that is no.? The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below.? I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>>? ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>>? ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>>? ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>>? typeid: INTEGER; >>>? ctype: REF CType; >> >>? ? ? ? ???^UNTRACED REF?? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>>? BuiltinUids{UID_INTEGER, &t_int}, >>>? BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &?? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what.? Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>>? ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>>? MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above.? Inconsistent >> number of levels of indirection.? A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >>? typeid: INTEGER; >>? ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int.? It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing.? Safe languages often >> make you stop and fix it before you run it.? Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development.? Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way.? E.g., the famous fake pointer to the root of a >> linked list example.? In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner.? If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:07:20 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:07:20 +0000 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: > If fact, if you are accustomed to thinking in type-safe terms I do think in fairly type-terms. In C++ I know to use templates, for example. There is a matter of lifetime-safety though. I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. Consider C++ code like this, where I want to do some things "the same" to a bunch of variables, and some things "specific". struct Person_t { const char* name; }; void F() { Person_t jay {"jay"}; Person_t randy = {"randy"}; Person_t tony = {"tony"}; // print all their names Person_t* people[] = {&jay, &randy, &tony}; for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i]->name); DoSomethingSpecificToJay(&jay); DoSomethingSpecificToRandy(&randy); DoSomethingSpecificToTony(&tony); } I do NOT want to do this: void F() { Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; // print all their names for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i].name); DoSomethingSpecificToJay(&people[0]); DoSomethingSpecificToRandy(&people[1]); DoSomethingSpecificToTony(&people[2]); } I understand there is: enum People { jay, randy, tony }; ... DoSomethingSpecificToJay(&people[jay]); DoSomethingSpecificToRandy(&people[randy]); DoSomethingSpecificToTony(&people[tony]); ... but I'm still not thrilled about that. I ended up without anything "specific" except initialization..so no really need to hang a name/variable on the data, and the initialization is fairly specific, so the resulting code is merely: EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); ... you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems in the m3cg interface...these types need to be defined via function calls into the backend, not hardcoded..and even if they are...the order of types arriving at the backend isn't always ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) - Jay > From: jay.krell at cornell.edu > Date: Wed, 15 Aug 2012 12:56:09 -0400 > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] reference to globals in globals? > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > Jay, > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/14/2012 10:04 PM, Jay K wrote: > >>> Isn't it safe to take the address of a global? > >>> > >> > >> Do you mean can't you use the ADR function in safe code > >> if you apply it only to a global variable? The answer > >> to that is no. The ADR function is illegal altogether in > >> safe code. > >> > >> As to why, I can only speculate, but see below. I suspect > >> even in this case, it is not as simple as it seems. > >> > >>> > >>> I have something like this: > >>> > >>> > >>> CONST UID_INTEGER = 1234; > >>> CONST UID_FLOAT = 4567; > >>> ... several more ... > >>> > >>> > >>> TYPE CType = OBJECT .. END; > >>> > >>> > >>> VAR t_int: CType := ...; > >>> VAR t_float: CType := ...; > >>> ... several more ... > >>> > >>> > >>> MapTypeIdToType(UID_INTEGER, t_int); > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > >>> ... several more ... > >>> > >>> > >>> but what I really want is more like: > >>> > >>> > >>> TYPE RECORD = BuiltinUid_t = > >>> typeid: INTEGER; > >>> ctype: REF CType; > >> > >> ^UNTRACED REF? If it were just REF, that would imply that > >> your global variable (the pointer it contains) is a heap object, that > >> it has heap allocator/GC overhead data attached to it, and that the GC > >> should trace it, none of which is true. > >> > >> > >>> END; > >>> > >>> > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > >>> BuiltinUids{UID_INTEGER, &t_int}, > >>> BuiltinUids{UID_FLOAT, &t_float}, > >> > >> ADR instead of &? If so, you are still not there, because ADR > >> returns a value of type ADDRESS, i.e., an untraced reference to > >> we-don't-know-what. Somewhere, you would also have to use a > >> LOOPHOLE to get it to UNTRACED REF CType. > >> > >>> ... several more ... > >>> }; > >>> > >>> > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > >>> END; > >>> > >> > >> I don't know what the signature of MapTypeIdToType is, but above, > >> you are passing a variable of object type to its 2nd parameter, > >> (which contains a traced reference to the actual heap object). > >> But here, you pass the _address_ of the above. Inconsistent > >> number of levels of indirection. A static safe language is > >> much more likely to help with things like this. > >> > >> Maybe you just want to say > >> > >> TYPE RECORD = BuiltinUid_t = > >> typeid: INTEGER; > >> ctype: CType; > >> > >> and > >> > >> BuiltinUids{UID_INTEGER, t_int}? > >> > >> This would be equivalent to your first way, and doesn't require any > >> unsafe coding at all. > >> > >> Or, you could do away with global variable t_int altogether and > >> just initialize directly into BuiltinUids[..].ctype with whatever > >> expression you used to initialize t_int. It looks like your array > >> makes the t_int and cousins redundant. > >> > >>> > >>> Heck, even if these weren't global, is it that unreasonble, > >>> from the programmer's point of view, for the language/compiler > >>> to do some pointer escape analysis and let me take the address > >>> of a local, as long as I don't store it somewhere that outlives > >>> the local? > >>> > >> > >> This is ultimately an undecidable problem and even conservative > >> approximations of reasonable sophistication are far too involved > >> for a language to require of every compiler. > >> > >>> > >>> You can see this particular pattern currently in > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > >>> > >>> > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > >>> where I encounter this. > >>> > >>> > >>> Working in safe languages can be frustrating... > >>> > >> > >> It's just an instant/deferred gratification thing. Safe languages often > >> make you stop and fix it before you run it. Unsafe languages let you naively > >> forge ahead to the next step, where the bug is likely to be *much* harder to > >> diagnose, assuming you even have enough test cases to notice it at all during > >> development. Your code here is a good example. > >> > >> Of course, safe languages occasionally make you unnecessarily write a bit more > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > >> linked list example. In my personal experience, these times are at least > >> one order of magnitude less frequent than the times safe languages reduce > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > >> first place. > >> > >> You're making it too difficult. > >> > >>> > >>> Thank you, > >>> - Jay > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:11:25 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:11:25 +0000 Subject: [M3devel] m3 backends and builtin types? Message-ID: Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:15:51 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:15:51 +0000 Subject: [M3devel] some proposed m3cg additions? Message-ID: I suggest backends should be informed of the following somehow via m3cg calls: word size definitely endian maybe jmpbuf size maybe target_name maybe I understand they are available from m3middle. Part of my agenda is that .mc files be more self contained. Eventually I'd like the IL to be independent of all of these anyway, but that is a later project. Jmpbuf size I know how to eliminate. Target_name doesn't really have meaning. Eliminating word size and endian dependencies in the IL won't be so easy. The extra information should be pretty harmless even if not critical. For example though...if you look at m3cgcat..it is slightly bad that it has to discover target from the ambient environment, instead of from the input file. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 01:58:23 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 19:58:23 -0400 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: Message-ID: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote: > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so, RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 16 02:55:12 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 15 Aug 2012 20:55:12 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: <20120816005512.GA31309@topoi.pooq.com> On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures? the ones whose environments are completely static? Or am I thinking of another type-safe language? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. You mean something like (in C with probable errors) list **p, *q; make q point to some linked list for ( p = &q; *p != NIL; p = &((*p)->next)) { And here you can easily delete elements by *p = (*p) -> next } This can be done prefectly type-safe in languages that allow REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik From dabenavidesd at yahoo.es Thu Aug 16 03:14:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 02:14:27 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> Message-ID: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html Thanks in advance --- El mi?, 15/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] reference to globals in globals? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 15 de agosto, 2012 19:55 On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures?? the ones whose environments are completely static? Or am I thinking of another type-safe language?? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way.? E.g., the famous fake pointer to the root of a > linked list example.? You mean something like (in C with probable errors) ? list **p, *q; ? make q point to some linked list ? for ( p = &q; *p != NIL; p = &((*p)->next)) ? { ???And here you can easily delete elements by *p = (*p) -> next ? } This can be done prefectly type-safe in languages that allow ???REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 04:07:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 02:07:52 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: > There is a bootstrap issue here isn?t there? I am not sure. I don't understand the system enough. :( I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. The compiler does for example construct the type BOOLEAN out of lower level pieces. Can it not then proceed to compute its typeid, and inform the backend about it? Other examples appear to include...wait.where did I get this list? At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). I built up the following list: UID_INTEGER = 16_195C2A74; (* INTEGER *) UID_LONGINT = 16_05562176; (* LONGINT *) UID_WORD = 16_97E237E2; (* CARDINAL *) UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) UID_REEL = 16_48E16572; (* REAL *) UID_LREEL = 16_94FE32F6; (* LONGREAL *) UID_XREEL = 16_9EE024E3; (* EXTENDED *) UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) UID_CHAR = 16_56E16863; (* CHAR [0..255] *) UID_WIDECHAR = 16_88F439FC; UID_MUTEX = 16_1541F475; (* MUTEX *) UID_TEXT = 16_50F86574; (* TEXT *) UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) UID_ROOT = 16_9D8FB489; (* ROOT *) UID_REFANY = 16_1C1C45E6; (* REFANY *) UID_ADDR = 16_08402063; (* ADDRESS *) UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) UID_NULL = 16_48EC756E; (* NULL *) Surely these aren't all fundamental?? Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: TYPE Record1 = RECORD REF Record2 END; TYPE Record2 = RECORD REF Record1 END; If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. - Jay From: hosking at cs.purdue.edu Date: Wed, 15 Aug 2012 19:58:23 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3 backends and builtin types? There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote:Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 06:23:37 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 04:23:37 +0000 Subject: [M3devel] m3middle/TFloat Message-ID: This TFloat module doesn't look right to me. I guess, ok, in a sense -- if host floating point is adequate.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 15:58:59 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 13:58:59 +0000 Subject: [M3devel] m3cg.set_target_name, set_endian, set_wordsize? Message-ID: I already said this, but more concretely, I propose M3CG_Ops additions: set_target_name(name: TEXT); TYPE Endian = { Little, Big }; (* or Endian_t or Endian_e *) set_endian(endian: Endian); set_wordsize(wordsize: INTEGER); (* currently always 32 or 64 *) I MIGHT have use of these in m3back/M3C. I'm not sure yet. target_name implies endian and wordsize and more, but it a kind of vague thing. It also implies win32 vs. Posix, which M3C probably won't need. It also implies jmpbuf size, which might be useful, or maybe we remove that "the other way". You know -- I want to eventually remove target knowledge from m3front, and have none in M3C either. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 16:21:01 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 14:21:01 +0000 Subject: [M3devel] higher level m3cg? Message-ID: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 17:06:18 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Thu, 16 Aug 2012 11:06:18 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Yes, I've thought so for a long time. It would also more easily enable llvm. Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 17:53:58 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 16:53:58 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345132438.30720.YahooMailClassic@web29701.mail.ird.yahoo.com> HI all: OO C is C++ in theory not in practice, could be better than this, but we should not do so and don't look a Modula-3 type system that preserves under compilation that's an abstract machine that interacts with whatever machine code generator is (M3CG) compiled to for whatever language you would like? That's why I insist the original acronym of M3CG(M2CG) is Machine Code Generator, this is not a translator, nor anything else, it was created for the purpose of compiling and linking compiler intermediate language, nothing else. Besides many languages and tongues I guess is better to speak clearly in a universal machine like Baby Modula-3, choosing Modula-3 as mother tongue, that is we need to speak in Modula-3 from Baby and counter wise. As some TV add said: "the problem is simple the answer it's not" So I would try to explain meaningfully clearly, we need to understand how to encode Modula-3 (1989) in a language like Quest (1994) in terms of Baby Modula-3 (1993), that is Target is BM3, Host is Quest, and Source is Modula-3. It turns out that doing type inference in terms of Quest is decidable, but in terms of Baby Modula-3 I don't know, it's pretty much a question of the million, the problem for real is can we encode Modula-3 in Baby Modula-3? That's a good question for the people who advocate for not so obscure language definition (It has some examples of understanding the language in terms of a exception key concept, but not in object terms) So, what is next is to find a theory of Objects, there are people that is not happy with Cardelli's definition and I agree with them we should we looking at the problem concretely with options I know of different kinds of options, but you would not like to hear the one I prefer is likely to be undecidable and inconsistent terms, so we could argue about obscurity of language definition again (right ?). This option is called pebble is a language written by Butler Lampson (Modula-2+ author) and Rodney Burstall (a aforementioned computer theorist ). In some sense Pebble is just about to talk about Modula-3 Module system, but it would need a lot of research to put it on place for being chosen. Thanks in advance --- El jue, 16/8/12, Jay K escribi?: De: Jay K Asunto: [M3devel] higher level m3cg? Para: "m3devel" Fecha: jueves, 16 de agosto, 2012 09:21 Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 16 18:40:45 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 11:40:45 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> References: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <502D228D.7070306@lcwb.coop> On 08/15/2012 08:14 PM, Daniel Alejandro Benavides D. wrote: > Hi all: > For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" > http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html > > Thanks in advance > I think here, "top" means outermost in the nested scopes of a compilation. That is, a "top-level" procedure is declared immediately inside an interface or module, not inside any procedure or block. This is an entirely different hierarchy from that of interfaces importing other interfaces, where the "bottom of the world" phrase is used to mean these types do not refer to any other imported interface. Oddly, I have heard this second sense referred to as the "top" of a type hierarchy, but that usage is highly counterintuitive to me. > --- El *mi?, 15/8/12, Hendrik Boom //* escribi?: > > > De: Hendrik Boom > Asunto: Re: [M3devel] reference to globals in globals? > Para: m3devel at elegosoft.com > Fecha: mi?rcoles, 15 de agosto, 2012 19:55 > > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > > > >Heck, even if these weren't global, is it that unreasonble, > > >from the programmer's point of view, for the language/compiler > > >to do some pointer escape analysis and let me take the address > > >of a local, as long as I don't store it somewhere that outlives > > >the local? > > > > > > > This is ultimately an undecidable problem and even conservative > > approximations of reasonable sophistication are far too involved > > for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > > > Of course, safe languages occasionally make you unnecessarily write a bit more > > code to do it the safe way. E.g., the famous fake pointer to the root of a > > linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } > > This can be done prefectly type-safe in languages that allow > > REF REF list > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Thu Aug 16 19:27:17 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 12:27:17 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502D2D75.8010702@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > Yes, yes, yes!!! > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? Yes. If you can assign a procedure (or pointer to function, which is equivalent, for this discussion) to a variable, then there are ways to call the procedure when its environment does not exist, i.e, one or more scopes of variables that the procedure is declared inside of and can refer to. The outermost scope always exists, so we need three nested scopes to get an example. MODULE Outer ; PROCEDURE Middle ( MiddleFormal : INTEGER ) = PROCEDURE Inner ( ) : INTEGER = BEGIN RETURN MiddleFormal END ; BEGIN (* Middle *) ProcVar := Inner (* Statically illegal because Inner is not top-level. *) END (* Middle ; VAR ProcVar : PROCEDURE ( ) : INTEGER ; VAR X : INTEGER ; BEGIN (* Outer *) Middle ( 7 ) ; X := ProcVar ( ) END Outer . If the assignment ProcVar := Inner were legal, the call ProcVar ( ) goes to Inner, which refers to MiddleFormal of Middle, but there is no active invocation of Middle, and MiddleFormal doesn't exist. Pascal (if I remember all this right) allows procedures to be passed as parameters, but not assigned to variables. The procedure constant in question can only be named where it has an environment. If all you can do with it is pass it, any use of it can only be deeper inside the dynamic call chain, where its environment continues to exist. Modula-2 allows procedures to be either passed or assigned, but in either case, only a top-level procedure, which always has an environment, can be given. Both passing and assigning procedures have reasonable uses, both for variations on the callback theme. Modula-3 supports both by allowing any procedure to be passed but only a top-level one to be assigned. Enforcing the latter restriction sometimes means a runtime check at the point of an assignment, if the RHS of the assignment is a procedure formal parameter, which could dynamically be top-level or not. This is probably not a high-frequency situation. Assigning a procedure constant and passing any procedure can be fully checked at compile time. I think it is possible to define a weaker, yet safe rule that postpones runtime environment existence checking until the point of a call on a procedure variable, but I don't recall seeing this in any language. Maybe Algol68 or something. In gcc-enhanced C, which allows nested functions, in addition to both assigning and passing (pointers to) functions, you are on your own to avoid calling something that lacks an environment. The one help the language gives is to say in the reference, "all hell will break loose" if you get this wrong. > > -- hendrik From rodney_bates at lcwb.coop Thu Aug 16 20:16:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 13:16:35 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502D3903.6020606@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: This is from memory, is incomplete, and my not be 100% reliable, but hopefully useful. There are two different hash codes on type structures used in (at least) the compiler, runtime, and pickles. The "uid" is 32-bit, and the "signature" is 64-bit. I think the signature maybe only applies to reference types. It is what is used in pickles. One of them takes into account some properties the other does not. I am not entirely sure which of the following information applies to which hash code(s). At one time, I think I had found a main program that computed one or the other of these, or both. They are not arbitrary, they are somehow derived from the actual type structures. (What are the type structures for primitive builtin types?) For builtin types, they have apparently been hand copied to several places. (compiler, RTS, pickles, m3gdb, ...?) They also get computed for programmer-defined types too, but obviously you won't find these hard-coded. However, some of the values for builtin types have suffered some alterations. Here is an excerpt from m3gdb, m3-lang.c, with a possibly relevant comment that I once inferred somehow: if (m3uid_to_num (uid, &num_uid)) { if (num_uid == 0x195c2a74) { return builtin_type_m3_integer; } else if (num_uid == 0x05562176) { return builtin_type_m3_longint; } else if (num_uid == 0x50f86574) { return builtin_type_m3_text; } else if (num_uid == 0x97e237e2) { return builtin_type_m3_cardinal; } else if (num_uid == 0x1e59237d) { return builtin_type_m3_boolean; } else if (num_uid == 0x08402063) { return builtin_type_m3_address; } else if (num_uid == 0x9d8fb489) { return builtin_type_m3_root; } else if (num_uid == 0xa973d3a6) { return builtin_type_m3_transient_root; } else if (num_uid == 0x56e16863) { return builtin_type_m3_char; } /* For widechar, the num_uid was once 0xb0830411. Presumably, this is an outdated leftover from a transitional implementation of widechar, at Critical Mass, before the final one came from them. */ else if (num_uid == 0x88f439fc) { return builtin_type_m3_widechar; } else if (num_uid == 0x48e16572) { return builtin_type_m3_real; } else if (num_uid == 0x94fe32f6) { return builtin_type_m3_longreal; } else if (num_uid == 0x9ee024e3) { return builtin_type_m3_extended; } else if (num_uid == 0x48ec756e) { return builtin_type_m3_null; } else if (num_uid == 0x1c1c45e6) { return builtin_type_m3_refany; } else if (num_uid == 0x51e4b739) { return builtin_type_m3_transient_refany; } else if (num_uid == 0x898ea789) { return builtin_type_m3_untraced_root; } else if (num_uid == 0x00000000) { return builtin_type_m3_void; } } Also, there is some endian-related confusion, at least for signatures. Sometimes they are stored in an array of bytes, whose order may be always the same, regardless of host or target. However, some were apparently hand-reordered between pm3 and cm3, and the reorderings are neither strictly endianness nor consistent for different types. I seem to recall something like swapping 32-bit segments, but not bytes within them, or the other way around, or something, for some but not all types. I had to deal with both versions of the orderings in pickles and maybe in m3gdb. Here is an excerpt from Pickle2.m3: (* Adaptation to some builtin fingerprints that have different byte order in Pm3 and Cm3. *) TYPE FPA = ARRAY [0..7] OF BITS 8 FOR [0..255]; (* Give a short name to anonymous type of Fingerprint.T.byte. *) CONST NULL_uid = 16_48ec756e; CONST pm3_NULL_Fp = FPA {16_24,16_80,16_00,16_00,16_6c,16_6c,16_75,16_6e}; CONST cm3_NULL_Fp = FPA {16_6e,16_75,16_6c,16_6c,16_00,16_00,16_80,16_24}; CONST ROOT_uid = 16_9d8fb489; CONST pm3_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_65,16_86,16_ad,16_41}; CONST cm3_ROOT_Fp = FPA {16_41,16_ad,16_86,16_65,16_c8,16_19,16_09,16_f8}; CONST UNTRACED_ROOT_uid = 16_898ea789; CONST pm3_UNTRACED_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_71,16_87,16_be,16_41}; CONST cm3_UNTRACED_ROOT_Fp = FPA {16_41,16_be,16_87,16_71,16_c8,16_19,16_09,16_f8}; (* Can the following two occur in a pickle? Maybe if somebody registered a special for them? *) CONST ADDRESS_uid = 16_08402063; CONST pm3_ADDRESS_Fp = FPA {16_91,16_21,16_8a,16_62,16_f2,16_01,16_ca,16_6a}; CONST cm3_ADDRESS_Fp = FPA {16_f2,16_01,16_ca,16_6a,16_91,16_21,16_8a,16_62}; CONST REFANY_uid = 16_1c1c45e6; CONST pm3_REFANY_Fp = FPA {16_65,16_72,16_24,16_80,16_79,16_6e,16_61,16_66}; CONST cm3_REFANY_Fp = FPA {16_66,16_61,16_6e,16_79,16_80,16_24,16_72,16_65}; Hope this is a step forward in the way of helpful information > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From jay.krell at cornell.edu Thu Aug 16 20:43:02 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 16 Aug 2012 14:43:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> References: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Message-ID: Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised". I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient. But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis. I have to figure out something for nested functions... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: > Yes, I've thought so for a long time. It would also more easily enable llvm. > > Sent from my iPhone > > On Aug 16, 2012, at 10:21, Jay K wrote: > >> Should m3cg provide enough information for a backend to generate idiomatic C? >> (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) >> >> >> Should we make it so? >> >> >> Or be pragmatic and see if anyone gets to that point? >> >> >> But, look at this another way. >> Let's say we are keeping the gcc backend. >> >> >> Isn't it reasonable to have a better experience with stock gdb? >> >> >> What should m3cg look like then? >> >> >> Matching up m3front to gcc turns out to be "wierd". >> As does having a backend generate "C". >> >> >> In particular, "wierd" because there is a "level mismatch". >> >> >> m3cg presents a fairly low level view of the program. >> It does layout. Global variables are stuffed into what you might call a "struct", with >> no assigned field names. Field references are done by adding to addresses and casting. >> >> >> Too low level to provide a "good" gcc tree representation or to generate "normal" C. >> >> >> One might be able to, by somewhat extraordinary means, make due. >> That is, specifically one could deduce field references from >> offsets/sizes. But maybe it is reasonable for load/store >> to include fields? Maybe in addition to what it provides? >> >> >> As well, it appears to me, that >> >> >> given TYPE Enum = {One, Two, Three}; >> >> the m3cg is like: >> >> declare enum typeidblah >> declare enum_elt One >> declare enum_elt Two >> declare enum_elt Three >> declare_typename typeidblah Enum >> >> >> One kind of instead wants more like: >> >> >> declare enum typeidblah Enum >> declare enum_elt One => rename it Enum_One >> declare enum_elt Two "" >> declare enum_elt Three "" >> >> >> However I understand that {One, Two, Three} exists >> as anonymous type independent of the name "Enum". >> >> >> One could just as well have: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = {One, Two, Three}; >> >> >> Enum1 and Enum2 probably have the same typeid, and are just >> two typenames for the same type. >> >> >> likewise: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = Enum1; >> >> >> but, pragmatically, in the interest of generating better C, >> can we pass a name along with declare_enum? >> >> I ask somewhat rhetorically. I realize there is the answer: >> enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >> typedef enum Mtypeid Enum1; >> >> >> Also, enum variables I believe end up as just UINT8, 16, or 32. >> Loads of enum values I believe end up as just loads of integers. >> Can we pass along optional enum names with declare_local/declare_param? >> And optional enum names with load_int? >> Or add a separate load_enum call? >> >> >> Really, I understand that the current interface can be pressed to do >> pretty adequate things. I can infer field references. The way enums work >> isn't too bad. >> >> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 23:13:55 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 22:13:55 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345151635.43876.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: Jay, even in RISC like G. Nelson mentioned there was this problem of big executables, and I happen to believe this doesn't change things too much for the better, so be careful with summing all kind of stuff will go on OK, there could be the side effects that we don't like too much. In the other side SPIN M3 Vortex compiler was a DEC-SRC modified front-end for that back-end to a Java-like subset backend, I don't know how well it was that, but at least they have done that already to the front-end so is less work to do. I don't care if you redefine the language to admittedly drop some unwanted changes but with some purpose of simplifying things, this was the best effort they did so we need to be aware of that. I generally don't take to heart this part of the system, but we need to respect the existence of this backend if somebody has a new better and brave CG that's better but to use your own language to do this I don't like the idea of several backends and different languages. Thanks in advance --- El jue, 16/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Antony Hosking" CC: "m3devel" , "Jay K" Fecha: jueves, 16 de agosto, 2012 13:43 Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised".? I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient.?But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis.I have to figure out something for nested functions... ?- Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: Yes, I've thought so for a long time. It would also more easily enable llvm.? Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 17 17:16:21 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 17 Aug 2012 11:16:21 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502D2D75.8010702@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> <502D2D75.8010702@lcwb.coop> Message-ID: <20120817151621.GB19487@topoi.pooq.com> On Thu, Aug 16, 2012 at 12:27:17PM -0500, Rodney M. Bates wrote: > > > On 08/15/2012 07:55 PM, Hendrik Boom wrote: > >On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > >> > >> > >>On 08/14/2012 10:04 PM, Jay K wrote: > >>> > >>>Heck, even if these weren't global, is it that unreasonble, > >>>from the programmer's point of view, for the language/compiler > >>>to do some pointer escape analysis and let me take the address > >>>of a local, as long as I don't store it somewhere that outlives > >>>the local? > >>> > >> > >>This is ultimately an undecidable problem and even conservative > >>approximations of reasonable sophistication are far too involved > >>for a language to require of every compiler. > > > >Not to mention the obscurityof the language definition. > > > > Yes, yes, yes!!! > > >But isn't there something like that with the addresses of top-level > >proocedures? the ones whose environments are completely static? > >Or am I thinking of another type-safe language? Or am I thinking of > >assigning the procedures themselves? > > Yes. If you can assign a procedure (or pointer to function, which is > equivalent, for this discussion) to a variable, then there are ways to > call the procedure when its environment does not exist, i.e, one or > more scopes of variables that the procedure is declared inside of and > can refer to. The outermost scope always exists, so we need three > nested scopes to get an example. > > MODULE Outer > > ; PROCEDURE Middle ( MiddleFormal : INTEGER ) > > = PROCEDURE Inner ( ) : INTEGER > > = BEGIN > RETURN MiddleFormal > END > > ; BEGIN (* Middle *) > ProcVar := Inner (* Statically illegal because Inner is not top-level. *) > END (* Middle > > ; VAR ProcVar : PROCEDURE ( ) : INTEGER > ; VAR X : INTEGER > > ; BEGIN (* Outer *) > Middle ( 7 ) > ; X := ProcVar ( ) > END Outer > . > > If the assignment ProcVar := Inner were legal, the call ProcVar ( ) > goes to Inner, which refers to MiddleFormal of Middle, but there is no > active invocation of Middle, and MiddleFormal doesn't exist. > > Pascal (if I remember all this right) allows procedures to be passed > as parameters, but not assigned to variables. The procedure constant > in question can only be named where it has an environment. If all you > can do with it is pass it, any use of it can only be deeper inside the > dynamic call chain, where its environment continues to exist. > > Modula-2 allows procedures to be either passed or assigned, but in > either case, only a top-level procedure, which always has an > environment, can be given. > > Both passing and assigning procedures have reasonable uses, both for > variations on the callback theme. > > Modula-3 supports both by allowing any procedure to be passed but only > a top-level one to be assigned. Enforcing the latter restriction > sometimes means a runtime check at the point of an assignment, if the > RHS of the assignment is a procedure formal parameter, which could > dynamically be top-level or not. This is probably not a > high-frequency situation. Assigning a procedure constant and passing > any procedure can be fully checked at compile time. > > I think it is possible to define a weaker, yet safe rule that > postpones runtime environment existence checking until the point of a > call on a procedure variable, but I don't recall seeing this in any > language. Maybe Algol68 or something. Algol 68 restricted assignment. The 'scope' of the procedure had to be at most as deeply nested as the 'scope' of the variable. This was a general restriction that applied to everything, though, not just procedures. This meant you could assign nested procedures to more nested variables. The 'scope' of a procedure was determined by its most local global variable. But scope checking led to heavy overhead on just about anything having to do with pointers, so it tended to be rarely inplemented. As on implementer said, this isn't the kind of mistake programmers make, anyway. The CDC Cyber implementation performed the check when called. Each scope (or rather, each scope for with this situation could arise) caused a small object to be allocated on the heap on scope entry. The stack frame would point to this object, and the object would point back to the stack frame. A procedure would contain a pointer to this object for the stak frame contining its most local globals.. WHen the procedure was called, it would check that this object indeed pointed to the place on the stack frame where there would be a pointer back to the object. Still, the restriction eliminated the possibility of writing a function-composition function. The only way I see of making this more liberal is to perform escape analysis on procedures, and subsequently choosing a few blocks whose stack frame would be allocated on the heap. You could apply run-time tests to prevent you from jumping into stack frames that have already been exited (of course in Modula 3, there aren't any GOTOs. It turns out that the analysis isn't all that difficult, and that for normal programs, almost no frames get allocated on the heap, hence little run-time overhead. The mechanism could be useful for callbacks. This is is the kind of analysis every decent Scheme compiler does to avoid accomodating the full force of conitinuation passing where it is not needed. -- hendrik > > In gcc-enhanced C, which allows nested functions, in addition to both > assigning and passing (pointers to) functions, you are on your own to > avoid calling something that lacks an environment. The one help the > language gives is to say in the reference, "all hell will break loose" > if you get this wrong. Does gcc-enhanced C allow you to put stack frames on the heap? -- hendrik From rodney_bates at lcwb.coop Fri Aug 17 20:51:12 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 13:51:12 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502E92A0.5080809@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } Hmm, this is not quite what I had in mind, but it's close. I like to code things like this with a pointer to the interesting node _and_ one that lags one node behind, in case list surgery is needed. Also, a more complete example would refer to one or more data fields of a node too. But I think I can illustrate my intentions with your example. > > This can be done prefectly type-safe in languages that allow > > REF REF list Plus, an ADR/& operation. In the case of C's &, it's type at least remembers the type of the object whose address is taken. Modula-3's ADR does not, so would require a LOOPHOLE as well. But no matter, because an operator that can take the address of anything other than a heap object (or maybe a component of one?) is unsafe. > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Fri Aug 17 21:12:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:12:35 -0500 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <502E97A3.2090804@lcwb.coop> On 08/16/2012 09:21 AM, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. In the stabs debug info given to (m3)gdb, they do have field names, with the original source code field name embedded within, plus a character-encoded uid and some flags, etc. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > This is the more reasonable representation, in view of the fact that any type expression is itself anonymous, as you note below, while giving it a type name is a separate matter, done outside the type definition. > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". Yes. > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > Yes. > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > This business of anonymous types is pervasive, not just for enumeration types and not just in Modula-3. It shows up all over the place in StubGen, which has lots of code obviously intended to handle anonymous object types and, not so obviously, broken, or at least unfinished. I guess nobody actually does it. Having once been inside that code, I would not define an anonymous type I wanted to have involved in anyway with network objects. Anonymous types are usually not particularly good coding style, and I almost always regret defining one and end up giving it a name after all, as I want to use it in multiple places. Modula-3's structural type equality rules perhaps encourage it more than so-called name equality. C encourages it too for the declarator types and by requiring manual construction of pass-by-reference by the programmer, instead of parameter modes. Something similar for the second-class treatment of arrays. > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > I don't remember the details, but m3gdb gets enumeration value names, displays them, and recognizes them in commands. They had to go through parse.c to get to stabs. > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > I know this is a broken record, but the fact that debug info comes off the side of parse.c means it can't reflect optimizations/transformations done later by the gcc backend. E.g. tree-nested.c. > - Jay From rodney_bates at lcwb.coop Fri Aug 17 21:28:53 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:28:53 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502E9B75.8040100@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive definitions and declarations, your M3 example being one. Modula-3's rule that all declarations in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, along with separate rules explicitly disallowing certain forms of recursive declarations that would be troublesome. E.g., deleting both occurrences of "REF" above would give records of infinite size. You might say the language uses "opt-out" for certain recursive declarations. Most languages use an opt-in strategy. The default rules is declare-before-use. For select kinds of declarations, there is some kind of "forward" declaration that declares the name and only some of its properties. After that, the name then can be used in limited ways (most interestingly, in other declarations). Eventually, there has to be a complete declaration that give the name again and supplies the remaining properties. After that it can be used in all legal ways. A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- use strategy with a system of forward declarations. My impression is that gcc is this way, but poorly documented. Will it give enough hints to just look at the front end output for selected recursive declaration sets like this example? It would at least give something known to be acceptable to gdb. Or is gdb changing the rules with new releases? > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From rodney_bates at lcwb.coop Sat Aug 18 19:51:11 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 18 Aug 2012 12:51:11 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: <502FD60F.7050002@lcwb.coop> On 08/15/2012 05:07 PM, Jay K wrote: > > If fact, if you are accustomed to thinking in type-safe terms > > > I do think in fairly type-terms. > In C++ I know to use templates, for example. > There is a matter of lifetime-safety though. > I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. > > > Consider C++ code like this, where I want to do some things > "the same" to a bunch of variables, and some things "specific". > > > struct Person_t { const char* name; }; > > void F() > { > Person_t jay {"jay"}; > Person_t randy = {"randy"}; > Person_t tony = {"tony"}; > > // print all their names > Person_t* people[] = {&jay, &randy, &tony}; > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i]->name); > > DoSomethingSpecificToJay(&jay); > DoSomethingSpecificToRandy(&randy); > DoSomethingSpecificToTony(&tony); > > } > In Modula-3, you can code like this too, but you have to declare it UNSAFE and use unsafe constructs, just like you have to use in C, but don't have to admit explicitly . > I do NOT want to do this: > > void F() > { > Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; > > > // print all their names > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i].name); > > DoSomethingSpecificToJay(&people[0]); > DoSomethingSpecificToRandy(&people[1]); > DoSomethingSpecificToTony(&people[2]); > > } > > I understand there is: > enum People { jay, randy, tony }; > ... > DoSomethingSpecificToJay(&people[jay]); > DoSomethingSpecificToRandy(&people[randy]); > DoSomethingSpecificToTony(&people[tony]); > ... > > > but I'm still not thrilled about that. > > I ended up without anything "specific" except initialization..so no > really need to hang a name/variable on the data, and the > initialization is fairly specific, so the resulting code is merely: > > > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); > > EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); > > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); > > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); > > ... > > you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems > in the m3cg interface...these types need to be defined via function calls into the backend, > not hardcoded..and even if they are...the order of types arriving at the backend isn't always > ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... > I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) > > > - Jay > > > > From: jay.krell at cornell.edu > > Date: Wed, 15 Aug 2012 12:56:09 -0400 > > To: hosking at cs.purdue.edu > > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] reference to globals in globals? > > > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > > > - Jay (briefly/pocket-sized-computer-aka-phone) > > > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > > > Jay, > > > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > > > >> > > >> > > >> On 08/14/2012 10:04 PM, Jay K wrote: > > >>> Isn't it safe to take the address of a global? > > >>> > > >> > > >> Do you mean can't you use the ADR function in safe code > > >> if you apply it only to a global variable? The answer > > >> to that is no. The ADR function is illegal altogether in > > >> safe code. > > >> > > >> As to why, I can only speculate, but see below. I suspect > > >> even in this case, it is not as simple as it seems. > > >> > > >>> > > >>> I have something like this: > > >>> > > >>> > > >>> CONST UID_INTEGER = 1234; > > >>> CONST UID_FLOAT = 4567; > > >>> ... several more ... > > >>> > > >>> > > >>> TYPE CType = OBJECT .. END; > > >>> > > >>> > > >>> VAR t_int: CType := ...; > > >>> VAR t_float: CType := ...; > > >>> ... several more ... > > >>> > > >>> > > >>> MapTypeIdToType(UID_INTEGER, t_int); > > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > > >>> ... several more ... > > >>> > > >>> > > >>> but what I really want is more like: > > >>> > > >>> > > >>> TYPE RECORD = BuiltinUid_t = > > >>> typeid: INTEGER; > > >>> ctype: REF CType; > > >> > > >> ^UNTRACED REF? If it were just REF, that would imply that > > >> your global variable (the pointer it contains) is a heap object, that > > >> it has heap allocator/GC overhead data attached to it, and that the GC > > >> should trace it, none of which is true. > > >> > > >> > > >>> END; > > >>> > > >>> > > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > > >>> BuiltinUids{UID_INTEGER, &t_int}, > > >>> BuiltinUids{UID_FLOAT, &t_float}, > > >> > > >> ADR instead of &? If so, you are still not there, because ADR > > >> returns a value of type ADDRESS, i.e., an untraced reference to > > >> we-don't-know-what. Somewhere, you would also have to use a > > >> LOOPHOLE to get it to UNTRACED REF CType. > > >> > > >>> ... several more ... > > >>> }; > > >>> > > >>> > > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > > >>> END; > > >>> > > >> > > >> I don't know what the signature of MapTypeIdToType is, but above, > > >> you are passing a variable of object type to its 2nd parameter, > > >> (which contains a traced reference to the actual heap object). > > >> But here, you pass the _address_ of the above. Inconsistent > > >> number of levels of indirection. A static safe language is > > >> much more likely to help with things like this. > > >> > > >> Maybe you just want to say > > >> > > >> TYPE RECORD = BuiltinUid_t = > > >> typeid: INTEGER; > > >> ctype: CType; > > >> > > >> and > > >> > > >> BuiltinUids{UID_INTEGER, t_int}? > > >> > > >> This would be equivalent to your first way, and doesn't require any > > >> unsafe coding at all. > > >> > > >> Or, you could do away with global variable t_int altogether and > > >> just initialize directly into BuiltinUids[..].ctype with whatever > > >> expression you used to initialize t_int. It looks like your array > > >> makes the t_int and cousins redundant. > > >> > > >>> > > >>> Heck, even if these weren't global, is it that unreasonble, > > >>> from the programmer's point of view, for the language/compiler > > >>> to do some pointer escape analysis and let me take the address > > >>> of a local, as long as I don't store it somewhere that outlives > > >>> the local? > > >>> > > >> > > >> This is ultimately an undecidable problem and even conservative > > >> approximations of reasonable sophistication are far too involved > > >> for a language to require of every compiler. > > >> > > >>> > > >>> You can see this particular pattern currently in > > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > >>> > > >>> > > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > > >>> where I encounter this. > > >>> > > >>> > > >>> Working in safe languages can be frustrating... > > >>> > > >> > > >> It's just an instant/deferred gratification thing. Safe languages often > > >> make you stop and fix it before you run it. Unsafe languages let you naively > > >> forge ahead to the next step, where the bug is likely to be *much* harder to > > >> diagnose, assuming you even have enough test cases to notice it at all during > > >> development. Your code here is a good example. > > >> > > >> Of course, safe languages occasionally make you unnecessarily write a bit more > > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > > >> linked list example. In my personal experience, these times are at least > > >> one order of magnitude less frequent than the times safe languages reduce > > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > > >> first place. > > >> > > >> You're making it too difficult. > > >> > > >>> > > >>> Thank you, > > >>> - Jay > > >> > > > From jay.krell at cornell.edu Sun Aug 19 09:11:30 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:11:30 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? Message-ID: I propose the following changed to m3front, TInt, TFloat. It does kind of "infact" the front/middle end with the "intent" of a specific backend. When a TInt or TFloat is initialized from text, that text is remembered. When such a TInt/TFloat is added/subtracted/multiplied/divided by another, the obvious text is built up in the target. Initialization of TInt with constants could/should also maintain the textual representation. The implication is multi part: 1) init_float in a C-or-such backend can just output the textual expression. No need to worry about roundtripping it. 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" (preceded by a typedef for INTEGER in the larger context)..thereby factoring out the target from the generated C. 3) This provides some hypothetical portability when host/target don't use the "same" sort of floating point. Ignoring rounding details however. And TFloat does seem to round correctly at compile-time, assuming host/target are about the same -- and they all are. (What does the language definition say here anyway? I'll have to check..) Reasonable? For now I'll probably punt. I can just use TFloat.ToChars and munge it slightly. But this is probably a good direction. In the short term I'm trying hard to make do w/o m3front/m3middle changes. You know..implementing another backend, you tend to visit more of the system, gradually understand it..think of what you how one might prefer it to be..esp. for the task at hand. :) Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be?If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 19 09:14:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:14:23 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <502E9B75.8040100@lcwb.coop> References: , , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu>, , <502E9B75.8040100@lcwb.coop> Message-ID: Thank for the pointers to m3gdb -- it kind of confirms my belief that the system isn't great.Even if there is a bootstrapping problem, it seems wrong for each backend and m3gdb to pay for it.Just m3front/m3front should have this hardcoded information. There is an m3front -emit option to ... i think provide the information..but it fails.I suspect it used and works when building just m3core.I have to look more. For now I'm punting on paying any attention to type information in the C-backend-in-progress.Just like the other backends ignore it all also.(ok..gcc uses it in the wierd way for m3gdb...) - Jay > Date: Fri, 17 Aug 2012 14:28:53 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > > > On 08/15/2012 09:07 PM, Jay K wrote: > > > There is a bootstrap issue here isn?t there? > > > > I am not sure. I don't understand the system enough. :( > > > > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > > The compiler does for example construct the type BOOLEAN out of lower level pieces. > > Can it not then proceed to compute its typeid, and inform the backend about it? > > > > > > Other examples appear to include...wait.where did I get this list? > > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > > I built up the following list: > > > > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > > UID_LONGINT = 16_05562176; (* LONGINT *) > > UID_WORD = 16_97E237E2; (* CARDINAL *) > > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > > UID_REEL = 16_48E16572; (* REAL *) > > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > > UID_WIDECHAR = 16_88F439FC; > > UID_MUTEX = 16_1541F475; (* MUTEX *) > > UID_TEXT = 16_50F86574; (* TEXT *) > > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > > UID_ROOT = 16_9D8FB489; (* ROOT *) > > UID_REFANY = 16_1C1C45E6; (* REFANY *) > > UID_ADDR = 16_08402063; (* ADDRESS *) > > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_NULL = 16_48EC756E; (* NULL *) > > > > > > Surely these aren't all fundamental?? > > > > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > > > > TYPE Record1 = RECORD REF Record2 END; > > TYPE Record2 = RECORD REF Record1 END; > > > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > > > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive > definitions and declarations, your M3 example being one. Modula-3's rule that all declarations > in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, > along with separate rules explicitly disallowing certain forms of recursive declarations > that would be troublesome. E.g., deleting both occurrences of "REF" above would give > records of infinite size. You might say the language uses "opt-out" for certain recursive > declarations. > > Most languages use an opt-in strategy. The default rules is declare-before-use. For select > kinds of declarations, there is some kind of "forward" declaration that declares the name > and only some of its properties. After that, the name then can be used in limited ways (most > interestingly, in other declarations). Eventually, there has to be a complete declaration that > give the name again and supplies the remaining properties. After that it can be used in all > legal ways. > > A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- > use strategy with a system of forward declarations. My impression is that gcc is this way, > but poorly documented. > > Will it give enough hints to just look at the front end output for selected recursive declaration > sets like this example? It would at least give something known to be acceptable to gdb. Or > is gdb changing the rules with new releases? > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > From: hosking at cs.purdue.edu > > Date: Wed, 15 Aug 2012 19:58:23 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3 backends and builtin types? > > > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > > > Something seems off to me in the current implementation. > > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > > I hardcoded knowledge of them in parse.c and M3C.m3. > > That seems wrong. > > > > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > > > > Shouldn't m3front inform the backend via m3cg of these types? > > It is doable using the existing interfaces? > > > > > > More so,RTBuiltin.mx ought not exist, right? > > Whatever data it contains should be built up like any other type data? > > Part of the same problem? > > > > > > - Jay > > > > > > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > > 305 N. University Street | West Lafayette | IN 47907 | USA > > Mobile+1 765 427 5484 > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 19 17:45:00 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 19 Aug 2012 11:45:00 -0400 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <20120819154500.GA17607@topoi.pooq.com> On Sun, Aug 19, 2012 at 07:11:30AM +0000, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. How many digits are you planning to keep for 1.0/3.0 ? -- hendrik From rodney_bates at lcwb.coop Sun Aug 19 23:48:27 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 19 Aug 2012 16:48:27 -0500 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <50315F2B.2060300@lcwb.coop> On 08/19/2012 02:11 AM, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > The implication is multi part: > > > 1) init_float in a C-or-such backend can just output the textual expression. > No need to worry about roundtripping it. > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > out the target from the generated C. > > > 3) This provides some hypothetical portability when host/target > don't use the "same" sort of floating point. Ignoring rounding details however. > And TFloat does seem to round correctly at compile-time, assuming host/target > are about the same -- and they all are. > (What does the language definition say here anyway? I'll have to check..) > As far as I can find, the language says nothing about this. However, I feel *extremely* strongly that arithmetic done at compile time should be *exactly* the same as on the target machine, including what happens in exceptional cases, e.g., overflows, NaNs, etc. I know this can be hard to get right, and it raises hard questions about division of work between front/back. Even harder when there are multiple back ends, especially ones you don't control. But I consider it a serious compiler bug if CT and RT arithmetic differ. This might be something that could be postponed for more urgent things, but the design at least should anticipate it. I have no problems with the principal of preserving/generating textual representations. However, if you really implement target arithmetic somewhere, it might be easier to convert back and forth than to do faithful target arithmetic on character strings. If that is what you are referring to as {not} roundtripping. > > Reasonable? > > > For now I'll probably punt. > I can just use TFloat.ToChars and munge it slightly. > But this is probably a good direction. > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > You know..implementing another backend, you tend to visit more of the system, > gradually understand it..think of what you how one might prefer it to be..esp. > for the task at hand. :) > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > - Jay > > From jay.krell at cornell.edu Mon Aug 20 00:08:26 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 22:08:26 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: <50315F2B.2060300@lcwb.coop> References: , <50315F2B.2060300@lcwb.coop> Message-ID: > However, if you really implement target arithmetic somewhere, I don't. I just want to print out constants or expressions for a C or C++ compiler to deal with. My initial goal is a C-generating backend that "work". The code will be target-dependent and not very debuggable (like what stock debuggers do today with Modula-3). Longer term, I want the code to be target-endian-indepenent, target-wordsize-independent, target-jmpbuf-independent, and possibly floating-point-representation-independent -- completely target-independent. Floating-point-representation independence doesn't gain anything -- all hosts and targets use the same representation, and I think the current system depends on that anyway (see TFloat). Floating-point-representation independence is "difficult" for "most compilers", but if generating C/C++, I think it is easy. target-wordsize-independence I believe shall be gained by my proposal here. target-endian-indepencence I believe shall be gained by small changes in m3front and the existing backends -- m3front only checks endianness in like one or two places. target-jmpbuf-size-independence I believe shall be gained by roughly the change I made in late 2010, but w/o causing loops to allocate stack for every try "invocation". Or, well, there is likely another way -- some indication to the backend that the variable is a jumpbuf and then outputing "jmp_buf jb" in the C backend..it should be easy enough..add declare_jmpbuf that has the same signature as declare_local, or add a boolean to declare_local. That is more efficient than the alloca(Csetjmp_jmpbuf_size) approach, but doesn't achieve the same gain for m3middle -- that is, even when targeting m3cc, m3front/m3middle ought not have this information -- that would cut out yet one more target-dependence in cm3/m3front/m3middle. Anyway, I'm not to this point yet. I have a fair amount more to do just for target-dependent C to be generated and work. I have made progress, but there is more to do. - Jay > Date: Sun, 19 Aug 2012 16:48:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? > > > > On 08/19/2012 02:11 AM, Jay K wrote: > > I propose the following changed to m3front, TInt, TFloat. > > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > > > > When a TInt or TFloat is initialized from text, that text is remembered. > > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > > the obvious text is built up in the target. > > > > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > > > > The implication is multi part: > > > > > > 1) init_float in a C-or-such backend can just output the textual expression. > > No need to worry about roundtripping it. > > > > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > > out the target from the generated C. > > > > > > 3) This provides some hypothetical portability when host/target > > don't use the "same" sort of floating point. Ignoring rounding details however. > > And TFloat does seem to round correctly at compile-time, assuming host/target > > are about the same -- and they all are. > > (What does the language definition say here anyway? I'll have to check..) > > > > As far as I can find, the language says nothing about this. > > However, I feel *extremely* strongly that arithmetic done at > compile time should be *exactly* the same as on the target machine, > including what happens in exceptional cases, e.g., overflows, > NaNs, etc. > > I know this can be hard to get right, and it raises hard questions > about division of work between front/back. Even harder when there > are multiple back ends, especially ones you don't control. > But I consider it a serious compiler bug if CT and RT arithmetic > differ. This might be something that could be postponed for more > urgent things, but the design at least should anticipate it. > > I have no problems with the principal of preserving/generating > textual representations. However, if you really implement target > arithmetic somewhere, it might be easier to convert back and forth > than to do faithful target arithmetic on character strings. If > that is what you are referring to as {not} roundtripping. > > > > > Reasonable? > > > > > > For now I'll probably punt. > > I can just use TFloat.ToChars and munge it slightly. > > But this is probably a good direction. > > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > > > > You know..implementing another backend, you tend to visit more of the system, > > gradually understand it..think of what you how one might prefer it to be..esp. > > for the task at hand. :) > > > > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > > > > - Jay > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Aug 21 13:18:48 2012 From: dmuysers at hotmail.com (Dirk Muysers) Date: Tue, 21 Aug 2012 13:18:48 +0200 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: *** A warning *** Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: Code generation is my business :-) Comments on a few options: a.. CLR: a.. Pro: industrial support b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter c.. Con: Only Windows platform is really prime-time quality b.. LLVM: a.. Pro: enthusiastic user community with charismatic leader b.. Pro: serious backing from Apple c.. Pro: many interesting performance improvements d.. Con: somewhat complex interface e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface c.. C-- a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code b.. Pro: design is reasonably mature and reasonably clean c.. Pro: supports accurate garbage collection d.. Pro: most users report it is very easy to use e.. Con: very small development team f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) g.. Con: does not ship with a garbage collector h.. Con: project has no future d.. C as target language a.. Pro: looks easy b.. Con: nearly impossible to get decent performance c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project. From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 14:33:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 13:33:27 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345552407.75520.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: It might need some pre-processor but C is supposedly superior in performance than C# and Java by far, I mean we are not talking about using C, we are talking about generating object code from UNSAFE code, but it shouldn't be that bad in terms of assembly level compile and link time process compared to those systems, if not then why every body wants their software be written in (e.g us) Modula-3, but compiled in a C environment it must be something about engineering causes of who wrote that thing first, and C is much older, so there is better support for that UNSAFE level code (instead look C3 and Java, lame), and that's for me more important in this matter. Probably C is referred as a high-level assembly (like DEC-SRC first versions), instead of a low-level assembly language like here we are trying to do. In fact if we were able to generate C from M3CG we can gain better portability, for instance my project of compare cross-boots would be more easily than by itself now. Thanks in advance --- El mar, 21/8/12, Dirk Muysers escribi?: De: Dirk Muysers Asunto: Re: [M3devel] higher level m3cg? Para: "Jay K" , "m3devel" Fecha: martes, 21 de agosto, 2012 06:18 *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 21 16:10:02 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Tue, 21 Aug 2012 10:10:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr. Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > CLR: > > Pro: industrial support > Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > Con: Only Windows platform is really prime-time quality > LLVM: > > Pro: enthusiastic user community with charismatic leader > Pro: serious backing from Apple > Pro: many interesting performance improvements > Con: somewhat complex interface > Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface > C-- > > Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > Pro: design is reasonably mature and reasonably clean > Pro: supports accurate garbage collection > Pro: most users report it is very easy to use > Con: very small development team > Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > Con: does not ship with a garbage collector > Con: project has no future > C as target language > > Pro: looks easy > Con: nearly impossible to get decent performance > Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 16:53:37 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 15:53:37 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <1345560817.79160.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: target dependency is essential in this process, so a good backend must target an quintessential machine, so I don't agree a given machine will give us lot of optimization without lot of work: http://irclog.perlgeek.de/parrot/2010-07-25#i_2605776 M3CG can be converted to a virtual machine as an emulator of an automaton, but studying that automaton complexity will give us the real facts (instead on believing stories), I don't want to fight wars among languages, but by practice we now know that C is far easier to code low level stuff than everything else, so .. if we want a radical change let's study why we do it first. Thanks in advance --- El mar, 21/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: "m3devel" , "Jay K" Fecha: martes, 21 de agosto, 2012 09:10 I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr.? Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Tue Aug 21 20:14:24 2012 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Tue, 21 Aug 2012 20:14:24 +0200 (CEST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: On Tue, 21 Aug 2012, Antony Hosking wrote: > *** A warning *** > Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: > ? > > Code generation is my business :-) > > Comments on a few options: > > * > > CLR: > > + Pro: industrial support > + Con: you have to buy into their type system pretty much completely; depending on what you want > to do with types, this may not matter > + Con: Only Windows platform is really prime-time quality > * > > LLVM: > > + Pro: enthusiastic user community with charismatic leader > + Pro: serious backing from Apple > + Pro: many interesting performance improvements > + Con: somewhat complex interface > + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > to be plugged by adding to the complexity of the interface Additional Con: They introduce bugs from time to time and then they do not fix them ... > * > > C-- > > + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > your C-- code LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > + Pro: design is reasonably mature and reasonably clean > + Pro: supports accurate garbage collection > + Pro: most users report it is very easy to use > + Con: very small development team > + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > + Con: does not ship with a garbage collector > + Con: project has no future > * > > C as target language > > + Pro: looks easy > + Con: nearly impossible to get decent performance > + Con: will drive you nuts in the long run; ask the long line of people who have tried to > compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > of these people gave up and built their own native code generator. > > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > and expected longevity, I'd probably recommend LLVM. Despite my 'con' point I would recommend LLVM, too. From jay.krell at cornell.edu Tue Aug 21 21:26:49 2012 From: jay.krell at cornell.edu (Jay) Date: Tue, 21 Aug 2012 12:26:49 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >> *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >> >> Code generation is my business :-) >> Comments on a few options: >> >> * >> CLR: >> >> + Pro: industrial support >> + Con: you have to buy into their type system pretty much completely; depending on what you want >> to do with types, this may not matter >> + Con: Only Windows platform is really prime-time quality >> * >> LLVM: >> >> + Pro: enthusiastic user community with charismatic leader >> + Pro: serious backing from Apple >> + Pro: many interesting performance improvements >> + Con: somewhat complex interface >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >> to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >> C-- >> >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >> your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >> + Pro: design is reasonably mature and reasonably clean >> + Pro: supports accurate garbage collection >> + Pro: most users report it is very easy to use >> + Con: very small development team >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >> + Con: does not ship with a garbage collector >> + Con: project has no future >> * >> C as target language >> >> + Pro: looks easy >> + Con: nearly impossible to get decent performance >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >> of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > From dabenavidesd at yahoo.es Tue Aug 21 23:58:52 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 22:58:52 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: <1345586332.94616.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: at the core of this issue, and if you want to ellucidate more, please C Ramsey, Lambda-RTL, and Necula Lambda-CIL respectively: http://terohasu.net/desclang/Description_languages.html#LAMBDA-RTL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.72.4008 Baby Modula-3 permits axiomatic reasoning over objects so it should be possible to reason over given program an its execution in a system of given semantics Thanks in advance --- El mar, 21/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Henning Thielemann" CC: "m3devel" , "JayK" Fecha: martes, 21 de agosto, 2012 14:26 I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >>? ? ? *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >>? >> Code generation is my business :-) >> Comments on a few options: >> >> * >>? ? CLR: >> >>? ???+? Pro: industrial support >>? ???+? Con: you have to buy into their type system pretty much completely; depending on what you want >>? ? ? ? to do with types, this may not matter >>? ???+? Con: Only Windows platform is really prime-time quality >> * >>? ? LLVM: >> >>? ???+? Pro: enthusiastic user community with charismatic leader >>? ???+? Pro: serious backing from Apple >>? ???+? Pro: many interesting performance improvements >>? ???+? Con: somewhat complex interface >>? ???+? Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >>? ? ? ? to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >>? ? C-- >> >>? ???+? Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >>? ? ? ? your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >>? ???+? Pro: design is reasonably mature and reasonably clean >>? ???+? Pro: supports accurate garbage collection >>? ???+? Pro: most users report it is very easy to use >>? ???+? Con: very small development team >>? ???+? Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >>? ???+? Con: does not ship with a garbage collector >>? ???+? Con: project has no future >> * >>? ? C as target language >> >>? ???+? Pro: looks easy >>? ???+? Con: nearly impossible to get decent performance >>? ???+? Con: will drive you nuts in the long run; ask the long line of people who have tried to >>? ? ? ? compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >>? ? ? ? of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 00:01:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Tue, 21 Aug 2012 22:01:23 +0000 Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> , <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: I find Ramsey's analysis rather incomplete. He mentions CLR has industrial strength support. But he fails to say that C also does, way more significantly so. And you can barely mention CLR w/o also mentioning Java, though CLR is more general. I know C has downsides. C, without C++, has the same poor exception handling. The performance of the compiler indeed will not be the best. The portability however will be tremendous. Basically every system I can think of except Windows Phone 7 and web browsers have a production quality C and C++ compiler. Examples of systems we don't support or support well today that a C/C++ backend would trivially enable, that are still manufactured, sold, updated: PPC{32,64}_AIX (still might need #ifdef in m3core) PPC64_{FREEBSD?,LINUX} ARM32_DARWIN (iPhone, iPad) MIPS64_{LINUX,OPENBSD} IA64_{VMS,HPUX,FREEBSD,LINUX} (still might need #ifdef in m3core due to the "two stacks"; I'm also worried that a 64bit integer closure marker is too small) ARM32_LINUX (including Android) MIPS32_LINUX (network routers!) ARM32_NT ARM64_? {PPC32,MIPS32,ARM32,I386}_CE plus some more obsolete systems: ALPHA{32,64}_VMS PPC64_DARWIN MIPS{32,64}_IRIX PA{32,64}_HPUX Granted, gcc/m3cc is *almost* as portable, and some of the above I have gotten far along in supporting -- really, given the gcc/m3cc/posix base, the biggest task to any port is getting the hardware up and running and with a decent C toolset. Plus that we would gain a "portable distribution format" that people could just tar xfz; cd; ./configure; make; make install Granted, I don't have much experience in any area here, and I don't yet have C working. LLVM will also take more time and learning. I probably won't put the time into it. I am hoping to get the C backend working, so that the cm3 system might be considered significantly more "done", so as new processors come along, no new porting will be needed, just as for the vast majority of code out there. LLVM supports many more architectures than listed below. Anyway..later.. - Jay > CC: hosking at cs.purdue.edu; dmuysers at hotmail.com; m3devel at elegosoft.com; jay.krell at cornell.edu > From: jay.krell at cornell.edu > Subject: Re: [M3devel] higher level m3cg? > Date: Tue, 21 Aug 2012 12:26:49 -0700 > To: lemming at henning-thielemann.de > > I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > > > > On Tue, 21 Aug 2012, Antony Hosking wrote: > > > >> *** A warning *** > >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > >> > >> Code generation is my business :-) > >> Comments on a few options: > >> > >> * > >> CLR: > >> > >> + Pro: industrial support > >> + Con: you have to buy into their type system pretty much completely; depending on what you want > >> to do with types, this may not matter > >> + Con: Only Windows platform is really prime-time quality > >> * > >> LLVM: > >> > >> + Pro: enthusiastic user community with charismatic leader > >> + Pro: serious backing from Apple > >> + Pro: many interesting performance improvements > >> + Con: somewhat complex interface > >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > >> to be plugged by adding to the complexity of the interface > > > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > > > > >> * > >> C-- > >> > >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > >> your C-- code > > > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > > > >> + Pro: design is reasonably mature and reasonably clean > >> + Pro: supports accurate garbage collection > >> + Pro: most users report it is very easy to use > >> + Con: very small development team > >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > >> + Con: does not ship with a garbage collector > >> + Con: project has no future > >> * > >> C as target language > >> > >> + Pro: looks easy > >> + Con: nearly impossible to get decent performance > >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to > >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > >> of these people gave up and built their own native code generator. > >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > >> and expected longevity, I'd probably recommend LLVM. > > > > Despite my 'con' point I would recommend LLVM, too. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 22 09:28:50 2012 From: wagner at elego.de (Olaf Wagner) Date: Wed, 22 Aug 2012 09:28:50 +0200 Subject: [M3devel] Fw: SW Bug #1080 Message-ID: <20120822092850.cef2e188.wagner@elego.de> I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system. I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From mika at async.caltech.edu Wed Aug 22 13:00:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 04:00:48 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822110048.903811A207D@async.async.caltech.edu> In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > > a.. CLR:=20 > > a.. Pro: industrial support=20 > b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader=20 > b.. Pro: serious backing from Apple=20 > c.. Pro: many interesting performance improvements=20 > d.. Con: somewhat complex interface=20 > e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 > b.. Pro: design is reasonably mature and reasonably clean=20 > c.. Pro: supports accurate garbage collection=20 > d.. Pro: most users report it is very easy to use=20 > e.. Con: very small development team=20 > f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 > g.. Con: does not ship with a garbage collector=20 > h.. Con: project has no future > d.. C as target language > > a.. Pro: looks easy=20 > b.. Con: nearly impossible to get decent performance=20 > c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 From jay.krell at cornell.edu Wed Aug 22 15:39:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:39:28 +0000 Subject: [M3devel] suggest fixed size integers in m3cg_binrd/binwr instead of compact encoding, for trivial decoding/looping? Message-ID: BinRd/BinWr "redesign". How about BinRd/BinWr is "redesigned" with the following in mind: instead of most compact encoding, instead use least-decoding-needed encoding, which is also least compact in particular, store all integers as native-endian 8 byte integers include at the start of the stream, like: UINT8 file_endian; UINT64 integer_map_offset; UINT64 integer_map_size; reader would mmap the file (mmap has its pluses and minuses). The data would be presumed to fit in address space -- even if it tends to be accessed sequentially, random access would be possible. if file_endian != host_endian, it would jump ahead to the positions indicated by integer_map_offset and swap them. writer would note all these offsets while writing If I made this change, then I'd be much more likely to just use this "pre-"existing" format as an in-memory format that I loop over multiple times. Given that these are temporary files that don't usually make it to disk, optimizing for size isn't clearly valuable. Given this change, walking the m3cg data multiple times would end up just being bumping a pointer through an array of variably-sized records, the sizes of which would be trivially looked up based on the first byte (with the exception of case_jump). This would likely obvious my ongoing little M3CG_MultiPass tedious work. Something I believe is desired/needed here for M3C.m3. This isn't meant to be a pointless excercise (though it is *somewhat* speculative). m3front/m3cg/m3cg_ops would be unchanged. The change would be in M3CG_BinRd.m3/M3CG_BinWr.m3/parse.c. Thoughts? I've omitted the obvious detail -- floats/doubles. There'd actually be an array of MultiByteValueMaps. They'd include a size. There'd be "4" for floats, and "8" for INTEGERs and doubles. And hypothetically 10 or 16 for "extended", but those don't really exist. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 15:36:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:36:28 +0000 Subject: [M3devel] m3cg "Proc" and "Var" Message-ID: I'm wondering if it might just be much easier if m3front assigns small incrementing integers for Var and Proc rather than getting pointers back from declare_procedure, declare_local, etc., and then passing them back to begin_procedure, load, etc. See, where I am at is I want to "playback" an m3cg run multiple times, computing different data each time -- later runs depending on earlier runs. I don't need to a persisent "disk" form. Though I end up close to that, esp. with this proposed change, which eliminates some pointers. I already do this in parse.c. Parse.c being written in C++ and having the preprocessor, it was much easier to do. See how I #include m3-def.h multiple times, with different macros. Ugly perhaps but highly convenient and productive and easy to use.. I'm so tempted to write "m3toc" this way, but I'm trying to use the product. It is painful.. Anyway, so I'm using Modula-3, and I have this really tedious code to convert the m3cg calls into an in-memory format. That I'll then loop over multiple times. Due to the tedium, I build up the in-memory format in one shared module, that I'm calling M3CG_MultiPass. It is closely analogous to M3CG_Ops.i3: TYPE declare_exception_t = REF RECORD name: Name; arg_typeid: TypeUID; raise_proc: BOOLEAN; base: Var; offset: INTEGER END; PROCEDURE set_runtime_proc(self: T; name: Name; proc: Proc) = BEGIN self.Add(NEW(set_runtime_proc_t, name := name, proc := proc)); END set_runtime_proc; etc. so, now, for this in-memory format to be used by arbitrary other backends..I think these "Var" and "Proc" will become a sticking point. I think we need a backend-independent way to refer to them. I think INTEGER is the way. Like how M3CG_BinWr deals with them. The downside/cost is that in M3x86.m3, rather than directly using the pointer (after narrow), it will instead index into its own private array. For efficiency, maybe some hint would come to the backend early as to how large to make the arrays? Or maybe that defeats some of the design -- the single pass implementations. (m3front is many-pass anyway, right? All this single-pass stuff is kind of only theoretical, right? The compiler working-set is not small and fixed size, but rather scales with module size, right?) Anyway, ok, either way. Backends can just at-least-double their internal arrays as they go. One might suggest some sort of "multiple playback" based on M3CG_BinWr/BinRd. I'm open to that. But it has some extra unnneeded overhead vs. a typical in-memory representation. I'm also open to changing BinWr/BinRd to use a larger representation that requires "no" decoding. (I've sketched this out myself, but decided it is not a priority. BinWr/BinRd really only exist for licensing reasons and would likely go away if m3cc went away. I think. They are kind of otherwise useful, at least during development.) Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 16:24:42 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:24:42 +0000 Subject: [M3devel] why this loop/multipass stuff Message-ID: why this loop/multipass stuff So people understand where I'm going here -- and Tony asked. I had already written this up.. 1) in the past working on m3cc/parse.c, I found that "declare"s weren't always "topologically sorted". That is, like, I might see: given TYPE T1 = INTEGER; TYPE T2 = RECORD = a:T1; END; I might have seen type T1 declared before T2, or not. I'm not sure if it was record/fields or other relationships. I just remember that building up a map of typeid to data, and looking up it when needed, didn't always work, with one pass. So what I did is that I was willing to "skip" declares that referred to typeids I hadn't seen yet, and then loop again, hoping for progress at each loop (fewer skips). 2) It looks like declare_local/temp don't all come in "at the start" of a procedure. I'm not sure if I'm generating C or C++, but if I'm generating C, it helps to have declare_local/temp right after begin_procedure/block. Multiple passes let me make that so, without changing m3front. (Upon further looking..it seems this is mostly done right, but not for "try" blocks; maybe reasonable to change m3front here.). Now, generating C++ is an option, and I'm ignoring all declares currently anyway, so these don't force the matter, but: 3) nested functions. If a function contains any nested functions, I want to put all of its locals in a local struct, and use the address of that struct as the "static link". Or at least the locals that are referenced by nested functions. As well, m3front contains options as to the order of outputing functions. It uses one order for m3back and a different order for m3cc. I wouldn't mind if M3C.m3 worked either way. There are possibly other benefits that could be derived from multiple passes. Nothing offhand though. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Aug 22 16:28:44 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 22 Aug 2012 10:28:44 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 22, 2012, at 10:24 AM, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Was this all in aid of building up proper type descriptors? > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. Preferably these only, or else you will cause all locals to be in memory instead of registers. > As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 22 16:41:09 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 07:41:09 -0700 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822144109.43DA71A2096@async.async.caltech.edu> Jay I've been following what you've been writing with one eye. But I'm not clear exactly what framework you're fitting into. Are you processing the output of m3front? There's always m3tk. Since it is claimed it was used to write a compiler it ought to be able to process MODULEs but I've only ever used it to process INTERFACEs. What's the story about that? Mika Tony Hosking writes: > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=iso-8859-1 > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > >> why this loop/multipass stuff >>=20 >>=20 >> So people understand where I'm going here -- and Tony asked. I had = >already written this up.. >>=20 >>=20 >>=20 >> 1) in the past working on m3cc/parse.c, I found that "declare"s >> weren't always "topologically sorted". >>=20 >>=20 >> That is, like, I might see: >>=20 >> given TYPE T1 =3D INTEGER; >> TYPE T2 =3D RECORD =3D a:T1; END; >>=20 >>=20 >> I might have seen type T1 declared before T2, or not. >> I'm not sure if it was record/fields or other relationships. >> I just remember that building up a map of typeid to data, and >> looking up it when needed, didn't always work, with one pass. >>=20 >>=20 >> So what I did is that I was willing to "skip" declares >> that referred to typeids I hadn't seen yet, and then loop >> again, hoping for progress at each loop (fewer skips). > >Was this all in aid of building up proper type descriptors? > >> 2) It looks like declare_local/temp don't all come in >> "at the start" of a procedure. I'm not sure if I'm generating >> C or C++, but if I'm generating C, it helps to have declare_local/temp >> right after begin_procedure/block. Multiple passes let me make that >> so, without changing m3front. >> (Upon further looking..it seems this is mostly done right, but >> not for "try" blocks; maybe reasonable to change m3front here.). >>=20 >>=20 >> Now, generating C++ is an option, and I'm ignoring all declares >> currently anyway, so these don't force the matter, but: >>=20 >>=20 >> 3) nested functions. >> If a function contains any nested functions, I want to put all of >> its locals in a local struct, and use the address of that struct >> as the "static link". Or at least the locals that are referenced >> by nested functions. > >Preferably these only, or else you will cause all locals to be in memory = >instead of registers. > >> As well, m3front contains options as to >> the order of outputing functions. It uses one order for m3back and >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either >> way. >>=20 >>=20 >> There are possibly other benefits that could be derived from multiple = >passes. >> Nothing offhand though. >>=20 >>=20 >> - Jay > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=iso-8859-1 > >charset=3Diso-8859-1">style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">
class=3D"Apple-style-span" color=3D"#0000FF">class=3D"Apple-style-span" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Antony = >Hoskingface=3D"Gill Sans">'Gill Sans'; ">'Gill Sans'; "> |class=3D"Apple-converted-space"> class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; ">class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = >">Associate Professorstyle=3D"font-family: 'Gill Sans'; ">style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = >University
face=3D"GillSans-Light">style=3D"font-family: GillSans-Light; ">305 N. University Street | West = >Lafayette | IN 47907 | USA
class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Mobileclass=3D"Apple-style-span" face=3D"GillSans-Light">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-converted-space"> +1 765 427 = >5484
face=3D"GillSans-Light">
class=3D"khtml-block-placeholder">
>

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline"> >
>
On Aug 22, 2012, at 10:24 AM, Jay K <href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = >wrote:

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">why this loop/multipass stuff


So people = >understand where I'm going here -- and Tony asked. I had already written = >this up..



1) in the past working on m3cc/parse.c, I found = >that "declare"s
weren't always "topologically = >sorted".


That is, like, I might see:

given TYPE T1 =3D = >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = >seen type T1 declared before T2, or not.
I'm not sure if it was = >record/fields or other relationships.
I just remember that building = >up a map of typeid to data, and
looking up it when needed, didn't = >always work, with one pass.


So what I did is that I was = >willing to "skip" declares
that referred to typeids I hadn't seen = >yet, and then loop
again, hoping for progress at each loop (fewer = >skips).

Was this all in = >aid of building up proper type descriptors?

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = >the start" of a procedure. I'm not sure if I'm generating
C or C++, = >but if I'm generating C, it helps to have declare_local/temp
right = >after begin_procedure/block. Multiple passes let me make that
so, = >without changing m3front.
(Upon further looking..it seems this is = >mostly done right, but
not for "try" blocks; maybe reasonable to = >change m3front here.).


Now, generating C++ is an option, and = >I'm ignoring all declares
currently anyway, so these don't force the = >matter, but:


3) nested functions.
If a function contains = >any nested functions, I want to put all of
its locals in a local = >struct, and use the address of that struct
as the "static link". Or = >at least the locals that are referenced
by nested = >functions.

Preferably these = >only, or else you will cause all locals to be in memory instead of = >registers.

style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = >">
As well, m3front contains options as to
the order = >of outputing functions. It uses one order for m3back and
a different = >order for m3cc. I wouldn't mind if M3C.m3 worked = >either
way.


There are possibly other benefits that could = >be derived from multiple passes.
Nothing offhand = >though.


 - = >Jay

= > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- From jay.krell at cornell.edu Wed Aug 22 16:56:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:56:31 +0000 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: <20120822144109.43DA71A2096@async.async.caltech.edu> References: , <20120822144109.43DA71A2096@async.async.caltech.edu> Message-ID: I'm implementing M3CG_Ops.m3. I don't trust m3tk -- it isn't the compiler. I'm somewhat familiar with m3cg. Given another M3CG, I know how to plug in to cm3, config files, etc. (e.g. add a new target, AMD64_NT, that will use this. :) ) On the other hand...I have a little more insight this morning. Variables have an "uplevel" parameter -- are they accessed from nested functions. I should be able to link variables to their containing procedure, and therefore its level. So "load(var)" I guess can be: text := "" for i := current_level to var.proc.level do (* typically an empty loop *) text := text & "_static_link->" end; if uplevel AND current_level = var.proc.level (* relatively rare *) text := text & "_locals."; end; text := text & var.name; or somesuch. I think I should get back to M3C with the assumption I don't need multiple passes...fil in more pieces..see where I get... - Jay > To: jay.krell at cornell.edu > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] why this loop/multipass stuff > Date: Wed, 22 Aug 2012 07:41:09 -0700 > From: mika at async.caltech.edu > > Jay I've been following what you've been writing with one eye. > > But I'm not clear exactly what framework you're fitting into. Are you > processing the output of m3front? > > There's always m3tk. Since it is claimed it was used to write a compiler > it ought to be able to process MODULEs but I've only ever used it to > process INTERFACEs. What's the story about that? > > Mika > > Tony Hosking writes: > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/plain; > > charset=iso-8859-1 > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > > > >> why this loop/multipass stuff > >>=20 > >>=20 > >> So people understand where I'm going here -- and Tony asked. I had = > >already written this up.. > >>=20 > >>=20 > >>=20 > >> 1) in the past working on m3cc/parse.c, I found that "declare"s > >> weren't always "topologically sorted". > >>=20 > >>=20 > >> That is, like, I might see: > >>=20 > >> given TYPE T1 =3D INTEGER; > >> TYPE T2 =3D RECORD =3D a:T1; END; > >>=20 > >>=20 > >> I might have seen type T1 declared before T2, or not. > >> I'm not sure if it was record/fields or other relationships. > >> I just remember that building up a map of typeid to data, and > >> looking up it when needed, didn't always work, with one pass. > >>=20 > >>=20 > >> So what I did is that I was willing to "skip" declares > >> that referred to typeids I hadn't seen yet, and then loop > >> again, hoping for progress at each loop (fewer skips). > > > >Was this all in aid of building up proper type descriptors? > > > >> 2) It looks like declare_local/temp don't all come in > >> "at the start" of a procedure. I'm not sure if I'm generating > >> C or C++, but if I'm generating C, it helps to have declare_local/temp > >> right after begin_procedure/block. Multiple passes let me make that > >> so, without changing m3front. > >> (Upon further looking..it seems this is mostly done right, but > >> not for "try" blocks; maybe reasonable to change m3front here.). > >>=20 > >>=20 > >> Now, generating C++ is an option, and I'm ignoring all declares > >> currently anyway, so these don't force the matter, but: > >>=20 > >>=20 > >> 3) nested functions. > >> If a function contains any nested functions, I want to put all of > >> its locals in a local struct, and use the address of that struct > >> as the "static link". Or at least the locals that are referenced > >> by nested functions. > > > >Preferably these only, or else you will cause all locals to be in memory = > >instead of registers. > > > >> As well, m3front contains options as to > >> the order of outputing functions. It uses one order for m3back and > >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either > >> way. > >>=20 > >>=20 > >> There are possibly other benefits that could be derived from multiple = > >passes. > >> Nothing offhand though. > >>=20 > >>=20 > >> - Jay > > > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/html; > > charset=iso-8859-1 > > > > >charset=3Diso-8859-1"> >style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; ">
> > >color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-border-horizontal-spacing: 0px; = > >-webkit-border-vertical-spacing: 0px; = > >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; font-size: medium; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = > >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = > >normal; font-variant: normal; font-weight: normal; letter-spacing: = > >normal; line-height: normal; orphans: 2; text-indent: 0px; = > >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; ">
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: = > >after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; ">
>class=3D"Apple-style-span" color=3D"#0000FF"> >class=3D"Apple-style-span" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Antony = > >Hosking >face=3D"Gill Sans"> >'Gill Sans'; "> >'Gill Sans'; "> | >class=3D"Apple-converted-space">  >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; "> >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = > >">Associate Professor >style=3D"font-family: 'Gill Sans'; "> >style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = > >University
> face=3D"GillSans-Light"> >style=3D"font-family: GillSans-Light; ">305 N. University Street | West = > >Lafayette | IN 47907 | USA
>class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Mobile >class=3D"Apple-style-span" face=3D"GillSans-Light"> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-converted-space"> +1 765 427 = > >5484
>face=3D"GillSans-Light">
>class=3D"khtml-block-placeholder">
>>

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline"> > >
> >
On Aug 22, 2012, at 10:24 AM, Jay K < >href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = > >wrote:

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">why this loop/multipass stuff


So people = > >understand where I'm going here -- and Tony asked. I had already written = > >this up..



1) in the past working on m3cc/parse.c, I found = > >that "declare"s
weren't always "topologically = > >sorted".


That is, like, I might see:

given TYPE T1 =3D = > >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = > >seen type T1 declared before T2, or not.
I'm not sure if it was = > >record/fields or other relationships.
I just remember that building = > >up a map of typeid to data, and
looking up it when needed, didn't = > >always work, with one pass.


So what I did is that I was = > >willing to "skip" declares
that referred to typeids I hadn't seen = > >yet, and then loop
again, hoping for progress at each loop (fewer = > >skips).

Was this all in = > >aid of building up proper type descriptors?

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = > >the start" of a procedure. I'm not sure if I'm generating
C or C++, = > >but if I'm generating C, it helps to have declare_local/temp
right = > >after begin_procedure/block. Multiple passes let me make that
so, = > >without changing m3front.
(Upon further looking..it seems this is = > >mostly done right, but
not for "try" blocks; maybe reasonable to = > >change m3front here.).


Now, generating C++ is an option, and = > >I'm ignoring all declares
currently anyway, so these don't force the = > >matter, but:


3) nested functions.
If a function contains = > >any nested functions, I want to put all of
its locals in a local = > >struct, and use the address of that struct
as the "static link". Or = > >at least the locals that are referenced
by nested = > >functions.

Preferably these = > >only, or else you will cause all locals to be in memory instead of = > >registers.

>style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = > >">
As well, m3front contains options as to
the order = > >of outputing functions. It uses one order for m3back and
a different = > >order for m3cc. I wouldn't mind if M3C.m3 worked = > >either
way.


There are possibly other benefits that could = > >be derived from multiple passes.
Nothing offhand = > >though.


 - = > >Jay

= > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 17:57:24 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 16:57:24 +0100 (BST) Subject: [M3devel] Fw: SW Bug #1080 In-Reply-To: <20120822092850.cef2e188.wagner@elego.de> Message-ID: <1345651044.13430.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: I'm on the process of booting a cm3-5.8.6-REL but in the second phase bootstrap process (NOPTHREAD) I have broke the compiler with this message (before I have fixed 2 quake thread scripts: m3-libs/m3core/src/thread/m3makefile:13 drop not clause m3-libs/m3core/src/thread.quake:87 same as above and add at line 90: else ??? PosixKernelThreads = FALSE ??? KernelThreads = FALSE ) : root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts# ./do-pkg.sh buildship m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 /home/test/code/cm3-5.8.6-REL/scripts/pkgmap.sh -c "/usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' " m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 === package /home/test/code/cm3-5.8.6-REL/m3-libs/m3core === ?+++ /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? +++ *** *** runtime error: ***??? Segmentation violation - possible attempt to dereference NIL ***??? pc = 0x8295954 = NoteStackLocations + 0x34 in ../src/runtime/common/RTCollector.m3 *** Aborted ?*** execution of /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? failed *** root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts#??? Antony, you have been busy but can you give us a clue on what to fix if so in m3core/src/runtime/common/RTCollector.m3 Thanks in advance --- El mi?, 22/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] Fw: SW Bug #1080 Para: m3devel at elegosoft.com CC: rob.zietlow at tdstelecom.com Fecha: mi?rcoles, 22 de agosto, 2012 02:28 I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL.? I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship???when I attempt to run cvsup right now it hangs.? When I connect to the cvsupd it also hangs.???I have strace access from the PIDS if needed.? One of the threads I came across commented about building with userthreads.? I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system.? I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 19:37:12 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 18:37:12 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822110048.903811A207D@async.async.caltech.edu> Message-ID: <1345657032.70947.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: in fact why not 'M3 as target language' is not the chicken and egg problem here, who needs C,? LLVM, stuff like that (we eat our own stuff and that's all), why not M3 in AST and that's all, like JSON la Olivetti. Olivetti tried to do that to compile and describe each node in the AST as an Object, we should explore the idea. http://lostechies.com/evanhoff/2008/01/08/language-roots/ Besides look for the 7th paragraph (I admire every day DEC-SRC and Modula-3 ever more) Thanks in advance --- El mi?, 22/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 06:00 In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? ? ? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; >??? charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > >? a.. CLR:=20 > >? ? a.. Pro: industrial support=20 >? ? b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 >? ? c.. Con: Only Windows platform is really prime-time quality >? b.. LLVM: > >? ? a.. Pro: enthusiastic user community with charismatic leader=20 >? ? b.. Pro: serious backing from Apple=20 >? ? c.. Pro: many interesting performance improvements=20 >? ? d.. Con: somewhat complex interface=20 >? ? e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface >? c.. C-- > >? ? a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 >? ? b.. Pro: design is reasonably mature and reasonably clean=20 >? ? c.. Pro: supports accurate garbage collection=20 >? ? d.. Pro: most users report it is very easy to use=20 >? ? e.. Con: very small development team=20 >? ? f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 >? ? g.. Con: does not ship with a garbage collector=20 >? ? h.. Con: project has no future >? d.. C as target language > >? ? a.. Pro: looks easy=20 >? ? b.. Con: nearly impossible to get decent performance=20 >? ? c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Wed Aug 22 20:11:06 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:11:06 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822181106.GA14178@topoi.pooq.com> On Wed, Aug 22, 2012 at 02:24:42PM +0000, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Does that mean you have a table of typeid's and their meanings in RAM during your multiple passes? If so, you might as well read them all in at once, and then link them together in RAM afterward. Or are you doing something sneakier? Or are you so short on RAM that thie is impractical? And for recorcds that are used by having pointers to them, you can just emit 'struct foo;' near the beginning of the generated C file. The topological sorting can be restricted to types that actually contain others, rather then pointing to them. This would reduce the amount of topological sorting you need to do. In fact, if you have all your types in RAM, you can do the topological sort *while* you are writing the types out. Just look through a type before emitting it and recurively emit any component types first. Mark each type when you emit it so you don't emit it twice. > > > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). How are you intending to make all these available for garbage collection? You might be interested in looking at Gambit/C, a Scheme compiler that compiles Schene to C and seems to do a fairly good job of it. > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. I wanted to do this in my Algol 68 compiler when I was still considering using the LLVM API, but placing temporaries and the like into the struct too so as to make it easier to form a struct descriptor for it for the garbage collector to ues. But that meant I'd be adding fields to it while I was gernerating the code that used it, and LLVM objected to it. Ecen though its whole API is oriented to putting pieces of the parse tree in piecemeal, jumping all arounf the thing if necessary, it still demanded that types be completely defined before you could use them for anything much. In particular, they had to be completely defined before you could use the API to generate parse tree that used the field names of the structure. This pretty well made the LLVM API significantly less attractive for this project. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay > From hendrik at topoi.pooq.com Wed Aug 22 20:38:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:38:52 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822183852.GB14178@topoi.pooq.com> On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > a.. CLR: > > a.. Pro: industrial support > b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader > b.. Pro: serious backing from Apple > c.. Pro: many interesting performance improvements > d.. Con: somewhat complex interface > e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead. Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to. Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again. If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish. This interfered with building run-time data structures whose full contents would not be known until compilation was complete. > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > b.. Pro: design is reasonably mature and reasonably clean > c.. Pro: supports accurate garbage collection > d.. Pro: most users report it is very easy to use > e.. Con: very small development team > f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > g.. Con: does not ship with a garbage collector > h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API. Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward. Of course this could be done with LLVM too. About the garbage collector. The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution. The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. > d.. C as target language > > a.. Pro: looks easy > b.. Con: nearly impossible to get decent performance > c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay From dabenavidesd at yahoo.es Thu Aug 23 02:51:46 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 01:51:46 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822183852.GB14178@topoi.pooq.com> Message-ID: <1345683106.74499.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I was reading an article and one a good consequence of Using a type safe like CLR, C-- would be be able to detect deadlocks with type inference. http://arxiv.org/pdarf/1002.0942.pdf So it can be good to do that for safety reasons, but at the cost of type inference time, which I think you wouldn't like too much for JIT compilers. Thanks in advance --- El mi?, 22/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] higher level m3cg? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 13:38 On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > >???a.. CLR: > >? ???a.. Pro: industrial support >? ???b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter >? ???c.. Con: Only Windows platform is really prime-time quality >???b.. LLVM: > >? ???a.. Pro: enthusiastic user community with charismatic leader >? ???b.. Pro: serious backing from Apple >? ???c.. Pro: many interesting performance improvements >? ???d.. Con: somewhat complex interface >? ???e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead.? Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to.? Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again.? If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish.? This interfered with building run-time data structures whose full contents would not be known until compilation was complete. ? >???c.. C-- > >? ???a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code >? ???b.. Pro: design is reasonably mature and reasonably clean >? ???c.. Pro: supports accurate garbage collection >? ???d.. Pro: most users report it is very easy to use >? ???e.. Con: very small development team >? ???f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >? ???g.. Con: does not ship with a garbage collector >? ???h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API.? Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward.? Of course this could be done with LLVM too. About the garbage collector.? The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution.? The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. >???d.. C as target language > >? ???a.. Pro: looks easy >? ???b.. Con: nearly impossible to get decent performance >? ???c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > >? > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. >???It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: >???enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >???typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > >? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 09:24:05 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 09:24:05 +0200 Subject: [M3devel] I know, I know... Message-ID: I know this will probably be very dense subject, but? What about: TSPacketHeader = <* ENDIAN = BIG*>RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) tErrInd, (* Transport Error Indicator *) pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) transPrio: BITS 1 FOR [0..1]; pid: BITS 13 FOR PID; transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) cc: BITS 4 FOR Nibble; END; Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: TSPacketHeader = RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) pidHi: BITS 5 FOR [16_00..16_1f]; transPrio: BITS 1 FOR [0..1]; pusi, tErrInd: BITS 1 FOR BOOLEAN; pidLo: BITS 8 FOR [16_00..16_ff]; cc: BITS 4 FOR Nibble; afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) END; And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. Thanks in advance.. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hosking at cs.purdue.edu Thu Aug 23 15:30:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 23 Aug 2012 09:30:57 -0400 Subject: [M3devel] I know, I know... In-Reply-To: References: Message-ID: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> PACKED? Sent from my iPad On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > I know this will probably be very dense subject, but? What about: > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > tErrInd, (* Transport Error Indicator *) > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > transPrio: BITS 1 FOR [0..1]; > pid: BITS 13 FOR PID; > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > cc: BITS 4 FOR Nibble; > END; > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > TSPacketHeader = RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > pidHi: BITS 5 FOR [16_00..16_1f]; > transPrio: BITS 1 FOR [0..1]; > pusi, > tErrInd: BITS 1 FOR BOOLEAN; > > pidLo: BITS 8 FOR [16_00..16_ff]; > > cc: BITS 4 FOR Nibble; > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > END; > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > Thanks in advance.. > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > From wagner at elego.de Thu Aug 23 16:24:03 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:24:03 +0200 Subject: [M3devel] CVSup hangs, was: Re: SW Bug #1080 In-Reply-To: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> References: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> Message-ID: <20120823162403.a2b0f83b.wagner@elego.de> On Wed, 22 Aug 2012 09:57:04 +0000 "Zietlow, Rob" wrote: > I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo > > I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. I think it might be helpful if you run the program in a debugger (gdb, m3gdb) and get a stack backtrace when it hangs, then post it to the list. Another approach might be to check if compilation with current sources will yield better results. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elego.de Thu Aug 23 16:33:19 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:33:19 +0200 Subject: [M3devel] recent and old bug reports Message-ID: <20120823163319.35c5bd77.wagner@elego.de> Is anybody out there with some time who could look into these issues: #1275 RTCollector problem when NOPTHREAD RT used libs 5.8.6 support wagner new 08/22/12 #1080 CVSUPD server hangs if used with -C option sys 5.8-RC3 sw-bug wagner new 12/29/09 #1148 CVSup stalls on several platforms misc 5.8.6 CM3 Release 5.9 sw-bug wagner new 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Thu Aug 23 17:25:28 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 17:25:28 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <3E9D81B8-0FC4-404A-A439-BEB21627F30E@m3w.org> packing, order, alignment. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 3:30 PM, Tony Hosking wrote: > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > >> I know this will probably be very dense subject, but? What about: >> >> TSPacketHeader = <* ENDIAN = BIG*>RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> tErrInd, (* Transport Error Indicator *) >> pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> transPrio: BITS 1 FOR [0..1]; >> pid: BITS 13 FOR PID; >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> cc: BITS 4 FOR Nibble; >> END; >> >> Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> >> Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> >> TSPacketHeader = RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> >> pidHi: BITS 5 FOR [16_00..16_1f]; >> transPrio: BITS 1 FOR [0..1]; >> pusi, >> tErrInd: BITS 1 FOR BOOLEAN; >> >> pidLo: BITS 8 FOR [16_00..16_ff]; >> >> cc: BITS 4 FOR Nibble; >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> END; >> >> And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> >> Thanks in advance.. >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 19:07:47 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:07:47 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741667.48136.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 23 19:08:03 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:08:03 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741683.90835.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:29:00 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:29:00 +0200 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> You mean, for system programming I have to use C? -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:26 PM, Jay K wrote: > How about you just use "bytes" or integers and helper functions like this: > > > typedef struct _BIT_FIELD { > UINT8 ByteOffset; > UINT8 BitOffset; > UINT8 BitCount; > } BIT_FIELD; > > > HRESULT > ExtractBitField(__in const BIT_FIELD* BitField, > __in const UINT8* Bytes, > __in size_t Size, > __out_opt UINT8* Value) > /*++ > Extract a bitfield from an array of bytes, as specified > by a byte offset, a bit offset within the first byte, and a bit count. > The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. > The bitfield can definitely cross byte boundaries. > --*/ > { > UINT8 const bitCount = BitField->BitCount; > UINT8 const bitOffset = BitField->BitOffset; > UINT8 const byteOffset = BitField->ByteOffset; > UINT8 const FF = (UINT8)~(UINT8)0; > BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); > UINT8 const secondBitOffset = (8 - bitOffset); > UINT8 localValue = { 0 }; > HRESULT error = { 0 }; > if (Value) > *Value = 0; > if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) > { > error = E_INVALIDARG; > goto Exit; > } > if ((byteOffset + twoBytes) >= Size) > { > error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); > goto Exit; > } > localValue = (Bytes[byteOffset] >> bitOffset); > if (twoBytes) > { > localValue &= ~(FF << secondBitOffset); > localValue |= (Bytes[byteOffset + 1] << secondBitOffset); > } > localValue &= ~(FF << bitCount); > error = 0; > if (Value) > *Value = localValue; > Exit: > return error; > } > > > arrays of bytes are really the only way to control the layout. > Look at how GNU binutils works, for example... > Bitfields in C don't yield predictable/portable layout either. > > > - Jay > > > From: hosking at cs.purdue.edu > > Date: Thu, 23 Aug 2012 09:30:57 -0400 > > To: dragisha at m3w.org > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] I know, I know... > > > > PACKED? > > > > Sent from my iPad > > > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > > > I know this will probably be very dense subject, but. What about: > > > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > tErrInd, (* Transport Error Indicator *) > > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > > transPrio: BITS 1 FOR [0..1]; > > > pid: BITS 13 FOR PID; > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > cc: BITS 4 FOR Nibble; > > > END; > > > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > > > TSPacketHeader = RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > > transPrio: BITS 1 FOR [0..1]; > > > pusi, > > > tErrInd: BITS 1 FOR BOOLEAN; > > > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > > > cc: BITS 4 FOR Nibble; > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > END; > > > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > > > Thanks in advance.. > > > -- > > > Divided by a common language > > > > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Thu Aug 23 21:26:58 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 19:26:58 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: How about you just use "bytes" or integers and helper functions like this: typedef struct _BIT_FIELD { UINT8 ByteOffset; UINT8 BitOffset; UINT8 BitCount; } BIT_FIELD; HRESULT ExtractBitField(__in const BIT_FIELD* BitField, __in const UINT8* Bytes, __in size_t Size, __out_opt UINT8* Value) /*++ Extract a bitfield from an array of bytes, as specified by a byte offset, a bit offset within the first byte, and a bit count. The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. The bitfield can definitely cross byte boundaries. --*/ { UINT8 const bitCount = BitField->BitCount; UINT8 const bitOffset = BitField->BitOffset; UINT8 const byteOffset = BitField->ByteOffset; UINT8 const FF = (UINT8)~(UINT8)0; BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); UINT8 const secondBitOffset = (8 - bitOffset); UINT8 localValue = { 0 }; HRESULT error = { 0 }; if (Value) *Value = 0; if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) { error = E_INVALIDARG; goto Exit; } if ((byteOffset + twoBytes) >= Size) { error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); goto Exit; } localValue = (Bytes[byteOffset] >> bitOffset); if (twoBytes) { localValue &= ~(FF << secondBitOffset); localValue |= (Bytes[byteOffset + 1] << secondBitOffset); } localValue &= ~(FF << bitCount); error = 0; if (Value) *Value = localValue; Exit: return error; } arrays of bytes are really the only way to control the layout.Look at how GNU binutils works, for example...Bitfields in C don't yield predictable/portable layout either. - Jay > From: hosking at cs.purdue.edu > Date: Thu, 23 Aug 2012 09:30:57 -0400 > To: dragisha at m3w.org > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > I know this will probably be very dense subject, but. What about: > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > tErrInd, (* Transport Error Indicator *) > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > transPrio: BITS 1 FOR [0..1]; > > pid: BITS 13 FOR PID; > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > cc: BITS 4 FOR Nibble; > > END; > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > TSPacketHeader = RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > transPrio: BITS 1 FOR [0..1]; > > pusi, > > tErrInd: BITS 1 FOR BOOLEAN; > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > cc: BITS 4 FOR Nibble; > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > END; > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > Thanks in advance.. > > -- > > Divided by a common language > > > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:42:51 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:42:51 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> Message-ID: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > You mean, for system programming I have to use C? > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:26 PM, Jay K wrote: > >> How about you just use "bytes" or integers and helper functions like this: >> >> >> typedef struct _BIT_FIELD { >> UINT8 ByteOffset; >> UINT8 BitOffset; >> UINT8 BitCount; >> } BIT_FIELD; >> >> >> HRESULT >> ExtractBitField(__in const BIT_FIELD* BitField, >> __in const UINT8* Bytes, >> __in size_t Size, >> __out_opt UINT8* Value) >> /*++ >> Extract a bitfield from an array of bytes, as specified >> by a byte offset, a bit offset within the first byte, and a bit count. >> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >> The bitfield can definitely cross byte boundaries. >> --*/ >> { >> UINT8 const bitCount = BitField->BitCount; >> UINT8 const bitOffset = BitField->BitOffset; >> UINT8 const byteOffset = BitField->ByteOffset; >> UINT8 const FF = (UINT8)~(UINT8)0; >> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >> UINT8 const secondBitOffset = (8 - bitOffset); >> UINT8 localValue = { 0 }; >> HRESULT error = { 0 }; >> if (Value) >> *Value = 0; >> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >> { >> error = E_INVALIDARG; >> goto Exit; >> } >> if ((byteOffset + twoBytes) >= Size) >> { >> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >> goto Exit; >> } >> localValue = (Bytes[byteOffset] >> bitOffset); >> if (twoBytes) >> { >> localValue &= ~(FF << secondBitOffset); >> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >> } >> localValue &= ~(FF << bitCount); >> error = 0; >> if (Value) >> *Value = localValue; >> Exit: >> return error; >> } >> >> >> arrays of bytes are really the only way to control the layout. >> Look at how GNU binutils works, for example... >> Bitfields in C don't yield predictable/portable layout either. >> >> >> - Jay >> >> > From: hosking at cs.purdue.edu >> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >> > To: dragisha at m3w.org >> > CC: m3devel at elegosoft.com >> > Subject: Re: [M3devel] I know, I know... >> > >> > PACKED? >> > >> > Sent from my iPad >> > >> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >> > >> > > I know this will probably be very dense subject, but. What about: >> > > >> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > tErrInd, (* Transport Error Indicator *) >> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> > > transPrio: BITS 1 FOR [0..1]; >> > > pid: BITS 13 FOR PID; >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > cc: BITS 4 FOR Nibble; >> > > END; >> > > >> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> > > >> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> > > >> > > TSPacketHeader = RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > >> > > pidHi: BITS 5 FOR [16_00..16_1f]; >> > > transPrio: BITS 1 FOR [0..1]; >> > > pusi, >> > > tErrInd: BITS 1 FOR BOOLEAN; >> > > >> > > pidLo: BITS 8 FOR [16_00..16_ff]; >> > > >> > > cc: BITS 4 FOR Nibble; >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > END; >> > > >> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> > > >> > > Thanks in advance.. >> > > -- >> > > Divided by a common language >> > > >> > > Dragi?a Duri? >> > > dragisha at m3w.org >> > > >> > > >> > > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Thu Aug 23 21:51:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 12:51:15 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: <20120823195115.80C121A2094@async.async.caltech.edu> Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" > > >--Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I have a friend, working for large software company and he just recently = >ported some message router from SPARC to Linux/Intel=85 Lots of "network = >order" data, same as with my current projects. Network, communication in = >general, network order is everywhere. > >Insisting on some god-given data ordering is a bit=85 What is nice word = >here? :) Outlandish? Outimeish? > >This is not first time I started discussion like this here. Every single = >time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >as I have shown in my example, with Modula-3 code. With a bit of effort = >I can make it almost-transparent (subfolder/platform) over various = >platforms. Of course, as I am developing a full product (not software to = >be run on arbitrary platform) I don't have to worry about too many = >platforms. One is enough here, but I still think Modula-3 can benefit, = >and a lot, if it supported explicit byte/bit ordering/packing/aligning = >pragmas. > >Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >we must write piece of software in C to be what? Compatible with API's = >all systems are compatible with without any hassle.=20 > >TIA >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From dragisha at m3w.org Thu Aug 23 21:59:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:59:46 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like? GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas of your own choice (or comments, or auxiliary data > from somewhere) and then generates code using Word.T that can process > byte streams and return the RECORD you declared, in portable Modula-3. > I doubt it's very difficult. m3tk is cool! > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" >> >> >> --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I have a friend, working for large software company and he just recently = >> ported some message router from SPARC to Linux/Intel=85 Lots of "network = >> order" data, same as with my current projects. Network, communication in = >> general, network order is everywhere. >> >> Insisting on some god-given data ordering is a bit=85 What is nice word = >> here? :) Outlandish? Outimeish? >> >> This is not first time I started discussion like this here. Every single = >> time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >> as I have shown in my example, with Modula-3 code. With a bit of effort = >> I can make it almost-transparent (subfolder/platform) over various = >> platforms. Of course, as I am developing a full product (not software to = >> be run on arbitrary platform) I don't have to worry about too many = >> platforms. One is enough here, but I still think Modula-3 can benefit, = >> and a lot, if it supported explicit byte/bit ordering/packing/aligning = >> pragmas. >> >> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >> we must write piece of software in C to be what? Compatible with API's = >> all systems are compatible with without any hassle.=20 >> >> TIA >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 22:34:09 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 21:34:09 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <2BADF386-2CFB-43EE-B809-F965AE351CD2@gmail.com> Message-ID: <1345754049.2762.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: just If you care for me, I don't have another machine to make bootstrap archives from newer compilers (that I don't have), and I haven't been following the trunck since when this stuff started to happen with CVSup (I think I still have the "mist stable RC3 or so"), so technically and morally, I have a dilemma, would need a lottery to make compilers work in so a long time when upgrading the compiler (maybe not). Manually I don't feel it's possible either as I lack the knowledge to do that either, so that's why I needed bootstrap archives but nobody answered here so... Thanks in advance --- El jue, 23/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] recent and old bug reports Para: "Daniel Alejandro Benavides D." CC: "m3devel at elegosoft.com" , "Olaf Wagner" Fecha: jueves, 23 de agosto, 2012 13:50 There are a number if bugs in the release that have since been fixed in the trunk. Best to upgrade from sources before trying to diagnose.? Sent from my iPhone On Aug 23, 2012, at 13:07, "Daniel Alejandro Benavides D." wrote: Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 00:12:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 22:12:48 +0000 Subject: [M3devel] I know, I know... In-Reply-To: References: , , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, Message-ID: I don't doubt that Modula-3 can handle it. And I don't doubt that C can handle it. But in both cases I find 1) the details beyond my attention span 2) definitely in the C case and very possibly in the Modula-3 case, not portable. You could learn the rules on one platform/compiler, only to produce code that doesn't work on others. Why bother learning them therefore? If you just want efficient storage in memory and don't care about the layout, then bitfields are ok. If you need to control the layout to interoperate with another system..even though bitfields are supposedly designed for this..I'm not sure I'd bother. C bitfields don't have portable layout. Do you suppose that Modula-3 ones do? Maybe there are in fact an obvious set of rules that all compilers follow? I don't know. Arrays of bytes have predictable layout. Look at how binutils reads/writes object files. It does like this: typedef struct elf_header_packed { char signature[4]; char architecture[4]; char offset_to_foo[4]; } elf_header_packed; typedef struct elf_header { long signature; long architecture; long offset_to_foo; } elf_header; and uses some pack/unpack functions to convert. They are solving a slightly different problem. They don't have bitfields necessarily to contend with, but just packing/size of integers that have a number of bits that are power of 2 (8, 16, 32, 64). - Jay From: dragisha at m3w.org Date: Thu, 23 Aug 2012 21:59:46 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] I know, I know... I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like. GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E Content-Type: multipart/alternative; boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1250 I have a friend, working for large software company and he just recently = ported some message router from SPARC to Linux/Intel=85 Lots of "network = order" data, same as with my current projects. Network, communication in = general, network order is everywhere. Insisting on some god-given data ordering is a bit=85 What is nice word = here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single = time Jay explains to me how Modula-3 cannot handle it. I am handling it, = as I have shown in my example, with Modula-3 code. With a bit of effort = I can make it almost-transparent (subfolder/platform) over various = platforms. Of course, as I am developing a full product (not software to = be run on arbitrary platform) I don't have to worry about too many = platforms. One is enough here, but I still think Modula-3 can benefit, = and a lot, if it supported explicit byte/bit ordering/packing/aligning = pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = we must write piece of software in C to be what? Compatible with API's = all systems are compatible with without any hassle.=20 TIA -- Divided by a common language Dragi=9Aa Duri=E6 dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Aug 24 01:07:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 16:07:44 -0700 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120823230744.509A61A2094@async.async.caltech.edu> Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before. If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline. Then I'd write a template file that allows me to specify said Scheme file in m3makefile. The advantage of this approach is that it doesn't rely on any compiler internals or special output files. And it's at least as powerful as any pragmas you might want to add. Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways). I only define two tables, you can imagine what the rest looks like. It's for a real-time financial trading application, if you were wondering. The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL. The code generator is only 1,197 lines of Scheme and that is including liberal documentation. (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.) The whole thing is added to the program by adding a single line to m3makefile. Mika (define aux-mm-ordr-ctrl (make-table "aux_mm_ordr_ctrl" 'client (make-field "symbol" 'varchar 'not-null) (make-index (list "symbol")) (make-field "type" 'varchar 'not-null) (make-field "expiry" 'varchar 'not-null) (make-field "exchange" 'varchar 'not-null) (make-field "currency" 'varchar 'not-null) (make-field "action" 'varchar 'not-null) (make-field "quantity" 'double-precision 'not-null) (make-field "order_type" 'varchar 'not-null) (make-field "benchmark_price" 'double-precision) (make-field "time_allowance" 'double-precision) (make-field "model_name" 'varchar) (make-index (list "model_name")) (make-field "id" 'integer) )) (define clean (make-table "clean" 'server (make-field "tabl" 'varchar 'not-null) (make-field "rowid" 'integer 'not-null) (make-field "client" 'varchar 'not-null) (make-index (list "client")) (make-index (list "tabl")) (make-index (list "tabl" "rowid")) (make-index (list "client" "tabl")))) (define gcoms (make-database "gcoms" clean code model instrument strategy model-data model-code ordr ordr_status output eror fill aux-instrument aux-ordr aux-ordr-params aux-market-segment aux-last-prices aux-mm-ordr-ctrl aux-mm-associations )) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future. And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult. m3tk is cool! >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- From jay.krell at cornell.edu Fri Aug 24 03:54:29 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 23 Aug 2012 18:54:29 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: Look at the code in m3core or libm3 that picks apart floats? The unsafe/loophole part isn't relevant. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 23, 2012, at 12:42 PM, Dragi?a Duri? wrote: > I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. > > Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? > > This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. > > Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. > > TIA > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > >> You mean, for system programming I have to use C? >> >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> >> On Aug 23, 2012, at 9:26 PM, Jay K wrote: >> >>> How about you just use "bytes" or integers and helper functions like this: >>> >>> >>> typedef struct _BIT_FIELD { >>> UINT8 ByteOffset; >>> UINT8 BitOffset; >>> UINT8 BitCount; >>> } BIT_FIELD; >>> >>> >>> HRESULT >>> ExtractBitField(__in const BIT_FIELD* BitField, >>> __in const UINT8* Bytes, >>> __in size_t Size, >>> __out_opt UINT8* Value) >>> /*++ >>> Extract a bitfield from an array of bytes, as specified >>> by a byte offset, a bit offset within the first byte, and a bit count. >>> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >>> The bitfield can definitely cross byte boundaries. >>> --*/ >>> { >>> UINT8 const bitCount = BitField->BitCount; >>> UINT8 const bitOffset = BitField->BitOffset; >>> UINT8 const byteOffset = BitField->ByteOffset; >>> UINT8 const FF = (UINT8)~(UINT8)0; >>> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >>> UINT8 const secondBitOffset = (8 - bitOffset); >>> UINT8 localValue = { 0 }; >>> HRESULT error = { 0 }; >>> if (Value) >>> *Value = 0; >>> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >>> { >>> error = E_INVALIDARG; >>> goto Exit; >>> } >>> if ((byteOffset + twoBytes) >= Size) >>> { >>> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >>> goto Exit; >>> } >>> localValue = (Bytes[byteOffset] >> bitOffset); >>> if (twoBytes) >>> { >>> localValue &= ~(FF << secondBitOffset); >>> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >>> } >>> localValue &= ~(FF << bitCount); >>> error = 0; >>> if (Value) >>> *Value = localValue; >>> Exit: >>> return error; >>> } >>> >>> >>> arrays of bytes are really the only way to control the layout. >>> Look at how GNU binutils works, for example... >>> Bitfields in C don't yield predictable/portable layout either. >>> >>> >>> - Jay >>> >>> > From: hosking at cs.purdue.edu >>> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >>> > To: dragisha at m3w.org >>> > CC: m3devel at elegosoft.com >>> > Subject: Re: [M3devel] I know, I know... >>> > >>> > PACKED? >>> > >>> > Sent from my iPad >>> > >>> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >>> > >>> > > I know this will probably be very dense subject, but. What about: >>> > > >>> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > tErrInd, (* Transport Error Indicator *) >>> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pid: BITS 13 FOR PID; >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > cc: BITS 4 FOR Nibble; >>> > > END; >>> > > >>> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >>> > > >>> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >>> > > >>> > > TSPacketHeader = RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > >>> > > pidHi: BITS 5 FOR [16_00..16_1f]; >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pusi, >>> > > tErrInd: BITS 1 FOR BOOLEAN; >>> > > >>> > > pidLo: BITS 8 FOR [16_00..16_ff]; >>> > > >>> > > cc: BITS 4 FOR Nibble; >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > END; >>> > > >>> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >>> > > >>> > > Thanks in advance.. >>> > > -- >>> > > Divided by a common language >>> > > >>> > > Dragi?a Duri? >>> > > dragisha at m3w.org >>> > > >>> > > >>> > > >>> > > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Aug 24 03:54:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 02:54:27 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120823230744.509A61A2094@async.async.caltech.edu> Message-ID: <1345773267.73869.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: No, I don't agree there isn't compiler internal intended to break interface that's other story but we don't fail compiled IRs at least that we know anything (at least the DEC-SRC front end needs to recompile every interface separately of any Module, so you can change compilers and still work the same clock speed should be equal in whatever configuration). We should take advantage of that inherent parallelism? I guess but we would need a good distributed quake machine. Alignment and endianness are same story about I don't mind that I just care for our actual clock speed of our machines instead of thinking in polymorphic type instructions etc. We eat our own stuff, VAX style machine type, everything else is just? head ache: http://mail-index.netbsd.org/current-users/1999/04/27/0032.html Thanks in advance --- El jue, 23/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] I know, I know... Para: "Dragi?a Duri?" CC: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 18:07 Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before.? If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline.? Then I'd write a template file that allows me to specify said Scheme file in m3makefile.? The advantage of this approach is that it doesn't rely on any compiler internals or special output files.? And it's at least as powerful as any pragmas you might want to add.? Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways).? I only define two tables, you can imagine what the rest looks like.? It's for a real-time financial trading application, if you were wondering.? The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL.? The code generator is only 1,197 lines of Scheme and that is including liberal documentation.? (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.)? The whole thing is added to the program by adding a single line to m3makefile. ? ???Mika (define aux-mm-ordr-ctrl ? (make-table "aux_mm_ordr_ctrl" 'client ? ? (make-field "symbol"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-index (list "symbol")) ? ? (make-field "type"? ? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "expiry"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "exchange"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "currency"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "action"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "quantity"? ? ? ? 'double-precision 'not-null) ? ? (make-field "order_type"? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "benchmark_price" 'double-precision) ? ? (make-field "time_allowance"? 'double-precision) ? ? (make-field "model_name"? ? ? 'varchar) ? ? (make-index (list "model_name")) ? ? (make-field "id"? ? ? ? ? ? ? 'integer) )) (define clean ? (make-table "clean" 'server ???(make-field "tabl"? ? ? ? ? 'varchar 'not-null) ???(make-field "rowid"? ? ? ???'integer 'not-null) ???(make-field "client"? ? ? ? 'varchar 'not-null) ? ? ???(make-index (list "client")) ???(make-index (list "tabl")) ???(make-index (list "tabl" "rowid")) ???(make-index (list "client" "tabl")))) ? ? (define gcoms ? (make-database "gcoms" ? ? ? ? ? ? ? ???clean ? ? ? ? ? ? ? ???code model instrument strategy model-data ? ? ? ? ? ? ? ???model-code ? ? ? ? ? ? ? ???ordr ordr_status ? ? ? ? ? ? ? ???output eror ? ? ? ? ? ? ? ???fill ? ? ? ? ? ? ? ???aux-instrument ? ? ? ? ? ? ? ???aux-ordr ? ? ? ? ? ? ? ???aux-ordr-params ? ? ? ? ? ? ? ???aux-market-segment ? ? ? ? ? ? ? ???aux-last-prices ? ? ? ? ? ? ? ???aux-mm-ordr-ctrl ? ? ? ? ? ? ? ???aux-mm-associations )) ? ? ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ??? (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; >??? boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; >??? charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future.? And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult.? m3tk is cool! >>=20 >>? ? Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> ??? boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> ??? charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; >??? charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; >??? filename=signature.asc >Content-Type: application/pgp-signature; >??? name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 24 16:00:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 24 Aug 2012 10:00:52 -0400 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120824140052.GA8309@topoi.pooq.com> On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice. If they are ignored on an implementation that doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone. Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away. It may be a deficiency we can live with. -- hendrik From dabenavidesd at yahoo.es Fri Aug 24 17:07:35 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 16:07:35 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> Message-ID: <1345820855.90388.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: In SPIN M3 you could VIEW any type as a bit string, but they also had a different compiler, so it's difficult to compare, in any case the thing written like that is UNSAFE in the data abstraction sense and so it's kind of easy see? that structure pattern in any other language with UNSAFE patterns. I like more structural typing in the sense if all are different you can't assign them, if you can then it's because they are not mismatched and obey the semantic rules of the language. Interesting combination of both would make a better RT though. Thanks in advance --- El vie, 24/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] I know, I know... Para: m3devel at elegosoft.com Fecha: viernes, 24 de agosto, 2012 09:00 On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future.? And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice.? If they are ignored on an implementation that? doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone.? Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away.? It may be a deficiency we can live with. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:43:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:43:44 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, <20120824140052.GA8309@topoi.pooq.com> Message-ID: Modula-3 has bit-mapping. Encoding it in functions that takes offsets and sizes is not hard. Using the language features gives you a choice of either - portable layout that doesn't likely match C - unportable layout that might match C At least unportable in terms of endianness -- again, for better and worse, see how we pick apart floats. It works, but the declarations are duplicated for little endian vs. big endian. Such knowledge does traditionally live in a compiler and its "core" libraries/runtime, but I'd still like to minimize it, maybe eliminate it. Neither is a ggeat choice I think. The compiler had tried to match target-specific layout/alignment rules. Whether it got them correct, I don't know. I relaxed the code because it was hard to know if it was right or wrong and just didn't seem worth maintaining -- i.e. porting to new systems. - Jay > Date: Fri, 24 Aug 2012 10:00:52 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > > (you can just use Word for that). > > > > But I think the point was that a compiler back-end might not necessarily > > obey your packing instructions? > > > > Here's what I think you should do... it would be super useful, both to > > you and other people in the future. And completely independent of > > compilers. > > > > Write a something-or-other that uses m3tk to read your RECORD type > > spec, plus pragmas > > Pragmas are definitely the wrong choice. If they are ignored on an > implementation that doesn't recognise them, this will be the wrong > semantics. > > Coding all the extraction and insertion functions is obscure, picky, and > error-prone. Bit-mapping is one of the reasons people use systems > languages, and the lack of it is a deficiency in Modula 3. > > Not that I'm advocating rewriting the compiler right away. It may be a > deficiency we can live with. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:48:02 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:48:02 +0000 Subject: [M3devel] Modula-3 bitfields Message-ID: Given: struct foo { union { struct { unsigned a : 16; unsigned b : 16; } s; unsigned c; } u; } foo = { {1, 2} }; printf("%X %X\n", c.u.s.a, c.u.s.b); What do people expect? Do people really have a mental model as to what the layout rules are for bitfields? Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? Furthermore, if presented with: struct foo { unsigned a : 16; unsigned b : 16; }; void F1(struct foo* f); printf("%X %X\n", f->a, f->b); Foo = RECORD a, b: BITS 16 FOR INTEGER END; <*EXTERNAL F1*> PROCEDURE F1(VAR foo: Foo); VAR f := Foo{1,2}; F1(f); What do people expect? That is, 1) Does anyone really internalize the layout rules for bitfields? 2) Does anyone really need them to line up with C? I ask because, it seems to me that in m3front/m3middle, if Little_endian were removed and assumed either always true or always false, everything would actually just work. The only thing that would go wrong is 1) interoperability with C..when there are bitfields 2) including in m3core where floats are picked apart 2b) But that's kind of good -- we could remove the little vs. big endian distinction there. See, I suspect but I am not certain the layout rules are actually well known to some small set of people, and they got encoded in m3front, in order for easier interoperability with C. Repeat the above exercises for more complicated examples, i.e. bitcounts that aren't multiples of 16. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri Aug 24 19:58:08 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 19:58:08 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: Message-ID: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> What mental model? We have network order. Everything network is bit streams, from left to right, most significant bits/bytes. No need for mental model here, any communication oriented ISO, ETSI, ITU standard? I don't understand why you insist on C syntax here. C has its own problems and reasons why it's at war with most things mental. Please refer to my initial example instead. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 7:48 PM, Jay K wrote: > printf("%X %X\n", c.u.s.a, c.u.s.b); > What do people expect? > Do people really have a mental model as to what the layout rules are for bitfields? > Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 20:10:35 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 11:10:35 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> Message-ID: <20120824181035.4EEFE1A207D@async.async.caltech.edu> The Green Book doesn't actually define alignment rules does it? Your example as I recall also specified precisely 32 bits. I can think of lots of ways things can go wrong. Why would you want to tie the hands of the compiler writer to a particular layout? Or clutter the interface with lots of pragmas... Yeah OK there was a day when that was the way to do things.... I think that time is past. The method I suggested could take advantage of situations where you know precisely how to construct the packed types. If you wanted. But it would also be entirely portable and independent of compilers. Heck you could generate C code at the same time and have free cross-language interoperability. (Or any other sets of languages.) You could also split bitfields in funny ways... Why would you do things any other way, really? Here's an example. A PCI Express DLLP is 32 bits of payload followed by a 16-bit CRC. Should I be able to write this as follows: DLLP = RECORD payload : BITS 32 FOR [0..16_ffffffff]; crc : BITS 16 FOR [0..16_ffff]; END; What then is the layout of ARRAY [0..1] OF DLLP; ? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Type: multipart/alternative; > boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >What mental model? > >We have network order. Everything network is bit streams, from left to = >right, most significant bits/bytes. No need for mental model here, any = >communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 > >I don't understand why you insist on C syntax here. C has its own = >problems and reasons why it's at war with most things mental. Please = >refer to my initial example instead. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 24, 2012, at 7:48 PM, Jay K wrote: > >> printf("%X %X\n", c.u.s.a, c.u.s.b); >> What do people expect? >> Do people really have a mental model as to what the layout rules are = >for bitfields? >> Are they obvious enough that every C compiler writer has actually = >implemented them the same and they become in a sense portable?=20 > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >mental model?

We have network order. Everything = >network is bit streams, from left to right, most significant bits/bytes. = >No need for mental model here, any communication oriented ISO, ETSI, ITU = >standard=E2=80=A6 

I don't understand why = >you insist on C syntax here. C has its own problems and reasons why it's = >at war with most things mental. Please refer to my initial example = >instead.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

class=3D"Apple-interchange-newline">
class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >expect?
Do people really have a mental model as to what the layout = >rules are for bitfields?
Are they obvious enough that every C = >compiler writer has actually implemented them the same and they become = >in a sense portable?class=3D"Apple-converted-space"> 
div>
= > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >=1GwZ >-----END PGP SIGNATURE----- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- From dragisha at m3w.org Fri Aug 24 21:20:03 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:20:03 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> As people in the world are divided in three groups, ones who can count and everybody else? So are we. I am in this group with this fancy idea of Modula-3 being system programming language. You don't agree? On Aug 24, 2012, at 8:10 PM, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Most probably. That's why I mentioned prgma in the first place. > > Your example as I recall also specified precisely 32 bits. > > I can think of lots of ways things can go wrong. > > Why would you want to tie the hands of the compiler writer to a particular > layout? Or clutter the interface with lots of pragmas? Pragma is good way to make things optional for some platforms, and to not change language definition. Platform specific, and working on AMD64 for example, is good enough. > > Yeah OK there was a day when that was the way to do things.... I think > that time is past. The method I suggested could take advantage of > situations where you know precisely how to construct the packed types. > If you wanted. But it would also be entirely portable and independent > of compilers. Heck you could generate C code at the same time and have > free cross-language interoperability. (Or any other sets of languages.) > > You could also split bitfields in funny ways... > > Why would you do things any other way, really? Being a Modula-2|3 programmer for 24 years obviously did funny things to me :). Like, I really started to like how high-level programs look like, esp ones doing various low-level stuff :). I like to think abstract, to write high-level, readable programs. Ones I don't have problems maintaining for prolonged time intervals. I don't need anything funny. I just need a way to represent, in a readable manner, standard communication data structures. So I can continue with my trade without C, > > Here's an example. A PCI Express DLLP is 32 bits of payload followed by > a 16-bit CRC. Should I be able to write this as follows: > > DLLP = RECORD > payload : BITS 32 FOR [0..16_ffffffff]; > crc : BITS 16 FOR [0..16_ffff]; > END; > > What then is the layout of > > ARRAY [0..1] OF DLLP; > Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? > ? > > Mika dd > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> What mental model? >> >> We have network order. Everything network is bit streams, from left to = >> right, most significant bits/bytes. No need for mental model here, any = >> communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 >> >> I don't understand why you insist on C syntax here. C has its own = >> problems and reasons why it's at war with most things mental. Please = >> refer to my initial example instead. >> >> -- >> Divided by a common language >> >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> >> On Aug 24, 2012, at 7:48 PM, Jay K wrote: >> >>> printf("%X %X\n", c.u.s.a, c.u.s.b); >>> What do people expect? >>> Do people really have a mental model as to what the layout rules are = >> for bitfields? >>> Are they obvious enough that every C compiler writer has actually = >> implemented them the same and they become in a sense portable?=20 >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >> mental model?

We have network order. Everything = >> network is bit streams, from left to right, most significant bits/bytes. = >> No need for mental model here, any communication oriented ISO, ETSI, ITU = >> standard=E2=80=A6 

I don't understand why = >> you insist on C syntax here. C has its own problems and reasons why it's = >> at war with most things mental. Please refer to my initial example = >> instead.

>> > color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">
--
Divided by a = >> common language

Dragi=C5=A1a = >> Duri=C4=87

= >>


> class=3D"Apple-interchange-newline"> >>
>>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

> class=3D"Apple-interchange-newline">
> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >> 16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >> expect?
Do people really have a mental model as to what the layout = >> rules are for bitfields?
Are they obvious enough that every C = >> compiler writer has actually implemented them the same and they become = >> in a sense portable?> class=3D"Apple-converted-space"> 
> div>
= >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Transfer-Encoding: 7bit >> Content-Disposition: attachment; >> filename=signature.asc >> Content-Type: application/pgp-signature; >> name=signature.asc >> Content-Description: Message signed with OpenPGP using GPGMail >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.18 (Darwin) >> >> iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >> fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >> tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >> 5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >> M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >> VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >> =1GwZ >> -----END PGP SIGNATURE----- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 21:28:59 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 12:28:59 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> Message-ID: <20120824192859.C3B801A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: ... >to me :). Like, I really started to like how high-level programs look = >like, esp ones doing various low-level stuff :). I like to think = >abstract, to write high-level, readable programs. Ones I don't have = >problems maintaining for prolonged time intervals. > > I don't need anything funny. I just need a way to represent, in a = >readable manner, standard communication data structures. So I can = >continue with my trade without C, > >>=20 >> Here's an example. A PCI Express DLLP is 32 bits of payload followed = >by >> a 16-bit CRC. Should I be able to write this as follows: >>=20 >> DLLP =3D RECORD >> payload : BITS 32 FOR [0..16_ffffffff]; >> crc : BITS 16 FOR [0..16_ffff]; >> END; >>=20 >> What then is the layout of=20 >>=20 >> ARRAY [0..1] OF DLLP; >>=20 > >Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika From dragisha at m3w.org Fri Aug 24 21:53:44 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:53:44 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824192859.C3B801A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> Message-ID: <14795BC4-4830-442C-A674-633197203234@m3w.org> What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: > > And what does that mean, exactly? No gaps allowed? What does it > mean, in particular, on a little-endian machine? > > What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > > What I am proposing isn't particularly difficult, it's not unportable, > either. And much more powerful than any sets of pragmas. Not tied to > your one architecture. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 05:06:06 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 03:06:06 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <14795BC4-4830-442C-A674-633197203234@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: > What mental model? > We have network order. Everything network is bit streams, from left to right, most > significant bits/bytes. Does any Modula-2 or Modula-3 or C compiler follow this model?For a little endian target? I'm skeptical.If it really is obvious, maybe we can make cm3 follow it?Even if it breaks bitfield interop with C?I would like to have m3front and M3C.m3 not have any target-endian dependentness.I do believe it would break pickles, and some floating point code in m3core/libm3. ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). In particular, standard C only allows bit fields to be of type "int" or "unsigned".However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? What does this look like: struct { unsigned a : 1; unsigned b : 32;};? - Jay From: dragisha at m3w.org Date: Fri, 24 Aug 2012 21:53:44 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Aug 25 07:35:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 22:35:51 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: <20120825053551.CB4BD1A207D@async.async.caltech.edu> Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. Mika Jay K writes: >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/plain; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > What mental model? > We have network order. Everything network is bit st= >reams=2C from left to right=2C most > significant bits/bytes. Does any Mod= >ula-2 or Modula-3 or C compiler follow this model?For a little endian targe= >t? I'm skeptical.If it really is obvious=2C maybe we can make cm3 follow it= >?Even if it breaks bitfield interop with C?I would like to have m3front and= > M3C.m3 not have any target-endian dependentness.I do believe it would brea= >k pickles=2C and some floating point code in m3core/libm3. ..because..you = >see.. I think there some mental model in C compiler writer's heads...that c= >m3 tries to adhere to=2C but which I might have diverged a bit from (look a= >t the history of Target.m3). In particular=2C standard C only allows bit f= >ields to be of type "int" or "unsigned".However it is a common extension to= > allow the type of a bitfield to also unsigned/signed char=2C short=2C long= >=2C and long long=2C and this does I believe affect the layout. Does that f= >it your mental model? In your model=2C must a 32bit bit field be aligned on= > a 32bit boundary? What does this look like: struct { unsigned a : 1=3B u= >nsigned b : 32=3B}=3B? - Jay > From: dragisha at m3w.org >Date: Fri=2C 24 Aug 2012 21:53:44 +0200 >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: Re: [M3devel] Modula-3 bitfields > >What you are proposing is to do a compiler-level job with a library. Just b= >ecause such a library exists does not mean it invalidates every other appro= >ach. And we are talking about one of most complicated pieces of Modula-3 sy= >stem=2C comparable in complexity with compiler. Of course=2C there are othe= >r approaches too=2C like some kind of interface language=2C to be processed= > into Modula-3 source code by cm3 build system. >Your latest example is good illustration for my proposal too. Another simpl= >e example to show a need for exact bit packing layout method. Thank you! >=0A= >--Divided by a common language >Dragi=B9a Duri=E6dragisha at m3w.org >=0A= >=0A= > >On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote: >And what does that mean=2C exactly? No gaps allowed? What does it >mean=2C in particular=2C on a little-endian machine? > >What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > >What I am proposing isn't particularly difficult=2C it's not unportable=2C >either. And much more powerful than any sets of pragmas. Not tied to >your one architecture. > > Mika > = > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/html; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > > >
 =3B>=3B What mental model= >?
 =3B>=3B We have network order. Everything network is bit s= >treams=2C from left to right=2C most
 =3B>=3B significant b= >its/bytes.
 =3B
 =3B
Does any Modula-= >2 or Modula-3 or C compiler follow this model?
For a little endia= >n target? I'm skeptical.
If it really is obvious=2C maybe we can = >make cm3 follow it?
Even if it breaks bitfield interop with C?iv>
I would like to have m3front and M3C.m3 not have any target-endian = >dependentness.
I do believe it would break pickles=2C and some fl= >oating point code in m3core/libm3.
 =3B ..because..you see.. = >I think there some mental model in C compiler writer's heads...that cm3 tri= >es to adhere to=2C but which I might have diverged a bit from (look at the = >history of Target.m3).
 =3B
 =3B
In p= >articular=2C standard C only allows bit fields to be of type "int" or "unsi= >gned".
However it is a common extension to allow the type of a bi= >tfield to also unsigned/signed char=2C short=2C long=2C and long long=2C an= >d this does I believe affect the layout. Does that fit your mental model? I= >n your model=2C must a 32bit bit field be aligned on a 32bit boundary?>
 =3B
What does this look like:
 =3B
= >
struct {
 =3B unsigned a : 1=3B
 =3B unsig= >ned b : 32=3B
}=3B
?
 =3B
 = >=3B
 =3B- Jay
 =3B
older">

From: dragisha at m3w.org
Date: Fri=2C= > 24 Aug 2012 21:53:44 +0200
To: mika at async.caltech.edu
CC: m3devel at el= >egosoft.com
Subject: Re: [M3devel] Modula-3 bitfields

What you ar= >e proposing is to do a compiler-level job with a library. Just because such= > a library exists does not mean it invalidates every other approach. And we= > are talking about one of most complicated pieces of Modula-3 system=2C com= >parable in complexity with compiler. Of course=2C there are other approache= >s too=2C like some kind of interface language=2C to be processed into Modul= >a-3 source code by cm3 build system.

Your latest ex= >ample is good illustration for my proposal too. Another simple example to s= >how a need for exact bit packing layout method. Thank you!

div>
=0A= >nt: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B white-space: norm= >al=3B border-collapse: separate=3B orphans: 2=3B widows: 2=3B" class=3D"ecx= >Apple-style-span"> none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B= > white-space: normal=3B border-collapse: separate=3B orphans: 2=3B widows: = >2=3B" class=3D"ecxApple-style-span">
>
--
Divided by a common language

Dr= >agi=B9a Duri=E6


newline">

=0A= >
=0A= >
On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote:
<= >br class=3D"ecxApple-interchange-newline">
ransform: none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacin= >g: 0px=3B white-space: normal=3B border-collapse: separate=3B orphans: 2=3B= > widows: 2=3B" class=3D"ecxApple-style-span">
And what does that mean=2C= > exactly?  =3BNo gaps allowed?  =3BWhat does it
mean=2C in parti= >cular=2C on a little-endian machine?

What about ARRAY [0..1] OF BITS= > 31 FOR [0..16_8fffffff] ?

What I am proposing isn't particularly di= >fficult=2C it's not unportable=2C
either.  =3BAnd much more powerful= > than any sets of pragmas.  =3BNot tied to
your one architecture.>
 =3B =3B =3B =3BMika

v>
>= > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_-- From dragisha at m3w.org Sat Aug 25 08:29:12 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:29:12 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: Did I said in my first post, pragma? With pragma you don't activate a behavior unless you need something pragma provides. Locally, in pragma's scope. So you will not break anything, anywhere outside pragma's scope. gm2 is very precise with bitpacking and aligning. I did not check for byte/bit order, or anything gm2 recently, too much work (most of it in Modula-3). As this is perfectly logical system programming requirement I do not doubt Gaius will be positive to such proposal. What good is bitfield interop when our pointers are 64bit aligned and GCC's are 32bit? And when every interop question is answered with "write critical piece in C" ? Proposed pragma would not break anything, it is local behavior. I believe in "mental model", but I also believe in evolution. Do you? :) All of us have decades of experience accumulated, and my decades changed my models, as slow as it is. I see this pattern, I see this need. I see solution through alternate bit/byte packing into structures inside pragma scope. As for your example, outside of proposed pragma scope it will be 32 bit aligned. But, if it is a 64 bit value - in GCC it will be 32bit aligned, while cm3 will (most probably) align it to 64bit. There goes interop. Ooops. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 5:06 AM, Jay K wrote: > > What mental model? > > We have network order. Everything network is bit streams, from left to right, most > > significant bits/bytes. > > > Does any Modula-2 or Modula-3 or C compiler follow this model? > For a little endian target? I'm skeptical. > If it really is obvious, maybe we can make cm3 follow it? > Even if it breaks bitfield interop with C? > I would like to have m3front and M3C.m3 not have any target-endian dependentness. > I do believe it would break pickles, and some floating point code in m3core/libm3. > ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). > > > In particular, standard C only allows bit fields to be of type "int" or "unsigned". > However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? > > What does this look like: > > struct { > unsigned a : 1; > unsigned b : 32; > }; > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sat Aug 25 08:31:16 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:31:16 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825053551.CB4BD1A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> Message-ID: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 09:22:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 07:22:44 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: 1) Most likely, whatever you want, you can get with Modula-3. You can at least attain the compactness you want. How you match up with some specified protocol, independently implementable e.g. with C and shifting/masking, less certain. 2) If this is for some network/persistant I/O, the result will not be portable. At least due to endianness. Possibly due to alignment. 3) I claim that interop with C remains a strong feature of Modula-3, but not necessarily regarding bitfields. There is some attempt at such interop, but I'd just as soon ignore it. I'd even give an error if types with bitfields are used with <* external *> functions. For my libc interop, I'm very conservative -- records contain just longint and integer, sorted by size -- there is never any padding, except maybe at the end. I looked a bit through "windows.i3" and "xlib.i3" and decided they seem pretty save too. 3) I am faced with generating target-independent C. This is the worry/doubt I inject/tangent into your line of questioning. You ask about one thing, I bring up something else (kind of like Daniel..) m3front checks endian, making it almost impossible. I fear there isn't a good answer here. I kind of would just like to a) remove the check b) adjust {libm3, m3core}/float accordingly. Alternatively, change m3front/m3cg some. Have the result say: #ifdef LITTLE_ENDIAN something #else something else #endif Perhaps the endian check can be removed if the target platform is "C"? Relevant aside: "By default", in C#, layout is undefined. It could sort the fields by size/name, I think, and be valid. Or put stuff near each other based on runtime profiling. Is Modula-3 this abstracted too?? But there are at least two ways to control the layout. You can assert that it must be "sequential". You can give actual offsets. I don't think C# even has bitfields though. - Jay Subject: Re: [M3devel] Modula-3 bitfields From: dragisha at m3w.org Date: Sat, 25 Aug 2012 08:31:16 +0200 CC: jay.krell at cornell.edu; m3devel at elegosoft.com To: mika at async.caltech.edu I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote:Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 13:22:17 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 13:22:17 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <21696995-DEE3-4DD7-AAB5-13C0EC62316B@m3w.org> On Aug 25, 2012, at 9:22 AM, Jay K wrote: > 1) Most likely, whatever you want, you can get with Modula-3. > You can at least attain the compactness you want. > How you match up with some specified protocol, independently > implementable e.g. with C and shifting/masking, less certain. Modula-3 can do everything needed, in ad-hoc and generally totally unportable manner :). > > > 2) If this is for some network/persistant I/O, the result will > not be portable. At least due to endianness. Possibly due to alignment. Not if we have means to force both endianness of data and alignment. > > > 3) I claim that interop with C remains a strong feature of Modula-3, > but not necessarily regarding bitfields. There is some attempt at such > interop, but I'd just as soon ignore it. I'd even give an error > if types with bitfields are used with <* external *> functions. I don't need much bitfield related interop with C. 32bit aligned pointers remain on my wish list :). > > > For my libc interop, I'm very conservative -- records contain just > longint and integer, sorted by size -- there is never any padding, > except maybe at the end. > I looked a bit through "windows.i3" and "xlib.i3" and decided > they seem pretty save too. libc interop is great, and I depend a lot on it. Thank you. > 3) I am faced with generating target-independent C. > This is the worry/doubt I inject/tangent into your line of questioning. > You ask about one thing, I bring up something else (kind of like Daniel..) :) This realy is a tangent. As my proposal was (localized to single structures) pragma, I don't see a link here. dd > m3front checks endian, making it almost impossible. > I fear there isn't a good answer here. > I kind of would just like to a) remove the check b) adjust > {libm3, m3core}/float accordingly. > Alternatively, change m3front/m3cg some. > Have the result say: > > > #ifdef LITTLE_ENDIAN > something > #else > something else > #endif > > > Perhaps the endian check can be removed if the target platform is "C"? > > > Relevant aside: > "By default", in C#, layout is undefined. > It could sort the fields by size/name, I think, and be valid. > Or put stuff near each other based on runtime profiling. > Is Modula-3 this abstracted too?? > But there are at least two ways to control the layout. > You can assert that it must be "sequential". > You can give actual offsets. > I don't think C# even has bitfields though. > > > - Jay > > > > > Subject: Re: [M3devel] Modula-3 bitfields > From: dragisha at m3w.org > Date: Sat, 25 Aug 2012 08:31:16 +0200 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: mika at async.caltech.edu > > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hendrik at topoi.pooq.com Sat Aug 25 21:05:48 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 25 Aug 2012 15:05:48 -0400 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <20120825190548.GA9036@topoi.pooq.com> On Sat, Aug 25, 2012 at 08:31:16AM +0200, Dragi?a Duri? wrote: > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to > C". Also - proposed pragma is local behavior. Not active unless you > need it, but when you need it you will be really grateful for it :). To adequately describe data structures defined elsewhere (as in communications standards) we need to be able to specify alignment, width, padding, and endianness. It doesn't need to be global for all the data in a program -- in fact, it had better not. It should be specified explicitly for each data structure or data item for which it matters. And it's a crucial part of the semantics of the program, not just something that the compiler should handle if it happens to recognise a pragma -- it needs to be part of the panguage proper. How the compiler chooses to store other data, which is not so draconically constrained, is entirely up to it. That said, it's still an open question whether to do something like this at all. But if it is to be done, it has to be done right. -- hendrik From microcode at zoho.com Sat Aug 25 21:13:17 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sat, 25 Aug 2012 19:13:17 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <20120825191317.GA3747@zoho.com> On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) From dragisha at m3w.org Sat Aug 25 22:08:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:08:46 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825191317.GA3747@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> Message-ID: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Systems Programming with Modula 3 Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote: > On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: >> The Green Book doesn't actually define alignment rules does it? > > Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison > but it's white and grimy (looks like it was a garage copy!) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 22:20:56 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 20:20:56 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: Microsoft C and gcc have a variety of ways to control this stuff. command line for the entire compilation #pragma pack/push/pop for a run of code __declspec(align) on individual types __unaligned on individual fields various __attribute__ probably on types and fields Which brings me back to ...array of bytes, very clear, very portable... If you need compactness but not exact layout, C and Modula-3 do fine. If you need a portable predictable layout, other than an array of bytes, they both largely fail. - Jay From: dragisha at m3w.org Date: Sat, 25 Aug 2012 22:08:46 +0200 To: microcode at zoho.com CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields Systems Programming with Modula 3Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote:On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 22:56:41 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:56:41 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> I am in no doubt about general inadequacies of C et al? What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 10:20 PM, Jay K wrote: > Which brings me back to ...array of bytes, very clear, very portable... > > If you need compactness but not exact layout, C and Modula-3 do fine. > If you need a portable predictable layout, other than an array of bytes, they both largely fail. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Aug 25 23:55:19 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 25 Aug 2012 14:55:19 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: <20120825215519.BA0431A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" > > >--Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I am in no doubt about general inadequacies of C et al=85 > >What I hate is an idea of setters and getters for my bitfields. With = >pragma, compiler generated load/store will behave as needed. Everything = >transparent and just-working. Until you have some data that is PDP-endian :-) Or want to split a field across a boundary that you're not supposed to be able to split a field across. Or have a field that's semantically scattered/gathered. Can your proposed scheme represent IEEE floating point? I agree it is a very useful idea, I agree it has to be done "right". But my definition of doing it "right" would be so demanding you'd be crazy to do it in the compiler.... Mika > >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From jay.krell at cornell.edu Sun Aug 26 04:59:26 2012 From: jay.krell at cornell.edu (Jay) Date: Sat, 25 Aug 2012 19:59:26 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: Other languages let you have "properties" that let you hide get/set functions behind field access syntax... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 25, 2012, at 1:56 PM, Dragi?a Duri? wrote: > I am in no doubt about general inadequacies of C et al? > > What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 10:20 PM, Jay K wrote: > >> Which brings me back to ...array of bytes, very clear, very portable... >> >> If you need compactness but not exact layout, C and Modula-3 do fine. >> If you need a portable predictable layout, other than an array of bytes, they both largely fail. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 26 06:56:59 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 26 Aug 2012 04:56:59 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <20120826045659.GA27610@zoho.com> On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- _ _ ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. From dragisha at m3w.org Sun Aug 26 09:03:14 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:03:14 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825215519.BA0431A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> <20120825215519.BA0431A207D@async.async.caltech.edu> Message-ID: <6AB35AF7-0D1A-40EA-B03A-E270B393CC8A@m3w.org> Right and simple are two faces of that coin. Every long walk has its first step. And so on :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 11:55 PM, Mika Nystrom wrote: > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" >> >> >> --Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I am in no doubt about general inadequacies of C et al=85 >> >> What I hate is an idea of setters and getters for my bitfields. With = >> pragma, compiler generated load/store will behave as needed. Everything = >> transparent and just-working. > > Until you have some data that is PDP-endian :-) > > Or want to split a field across a boundary that you're not supposed to > be able to split a field across. > > Or have a field that's semantically scattered/gathered. > > Can your proposed scheme represent IEEE floating point? > > I agree it is a very useful idea, I agree it has to be done "right". > But my definition of doing it "right" would be so demanding you'd be > crazy to do it in the compiler.... > > Mika > >> >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sun Aug 26 09:05:57 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:05:57 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <20120826045659.GA27610@zoho.com> Message-ID: <634D1148-0033-4037-A20E-AAAD05F0CFF1@m3w.org> Books don't disappear, usually :). They go out of print and move to afterlife in antiquity shops and forgotten shelves :). Google for table of contest of SPwM3 and match with names of SRC reports. I don't have a book with me right now so I can't write it out for you, sorry. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 26, 2012, at 6:56 AM, microcode at zoho.com wrote: > On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >> >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers) >> > > Oh thanks. I'll try to get a copy of that if it is good. I did download most > of the SRC stuff but have no idea when I'll have time to look at it. > > I'm trying to get a few books before they all disappear. > > -- > _ _ > ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ > | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> > |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Mon Aug 27 04:18:29 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 02:18:29 +0000 Subject: [M3devel] multiple evaluation hazard in backend? Message-ID: Tony, does m3front tend or tend not to ask the backend to evaluate the same expression multiple times, or does it tend to allocate a temporary if it would do so? It's just that my current approach, if not given temporaries, may tend to evaluate expressions multiple times that either must not (for correctness) or ought not (for efficiency). I guess I'll learn soon enough. My current "stack" is just TEXTs. Very simple. If I had to eliminate multiple evaluations I'd have to consider a few alternatives: 1) for every, push, do something like "{ T s0 = expr;" for every pop "}" That would require I always "use before pop", not "use not after pop" as I tend do. That isn't always easy. The above is also wrong. Instead of "s0", it'd be "sN" where N is the current stack depth, then "get(n)" would, like, get N-n. 2) Alternatively, if I knew the maximum stack depth such as by making two passes, I could have an array of unions. That would make casts less verbose -- they'd be via union member access, if no size change. 2b) I could maintain a stack pointer at compile time or runtime. Runtime easier but slower. I still might want to record types with my texts, like to avoid extra casts. Or maybe m3front does that already? Anyway, for now I can ignore it. I'm just starting in on function calls, and I still haven't handled nested functions. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 04:37:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 03:37:31 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> Message-ID: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email and I'll tell you url) in a public site, SPwM3 is almost but last chapter is presumably in other hands (the so-called lost video tapes), and the other two books are (OK are not totally on-line, but bigger part are available from archive.org). I was trying hard to compile SPwM3 in one file, but the more I see it, I guess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was already available, we have already the C.1 - Introduction, C. 2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An? Introduction to programming with Threads (available from MS-Research in C# syntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are almost in source tree, it will just miss C. 5 - Thread Synchronization (but is one of the smallest < 20 p. I believe we can make a revised copy based on current Interface specification after we get a copy of ESC/M3). It needs a bit of work I guess, but if some kind soul is wanting to do this I will let him do it. The only needed book is Algorithms in Modula-3 (but there is a German script of very good quality from U Aachen) so, it's not too bad but sure will be nice to recover it from somewhere (I heard it was used to develop complete Algorithm Animations, which would be perfect for Mentor) Thanks in advance --- El s?b, 25/8/12, microcode at zoho.com escribi?: De: microcode at zoho.com Asunto: Re: [M3devel] Modula-3 bitfields Para: m3devel at elegosoft.com Fecha: s?bado, 25 de agosto, 2012 23:56 On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- ? ? ? ? _? ? ? ? ? ? ? ? ? ? ? ? ? ???_? ? ? ._ _ _ <_> ___? _ _? ___? ___? ___? _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_|? \___/\_|_.\___/\___|\___. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 05:08:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 03:08:31 +0000 Subject: [M3devel] race conditions in lock-free code... Message-ID: We have race conditions, like this, m3core/src/Text.m3: VAR fromCharCache := ARRAY CHAR OF T {NIL, ..}; (* 1-char texts *) PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; fromCharCache[c] := Text8.New (buf); END; RETURN fromCharCache [c] END FromChar; It should be: PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; WITH a = Text8.New (buf) DO MemoryBarrier(); fromCharCache[c] := a; END; END; RETURN fromCharCache [c] END FromChar; to ensure that all of Text8.New() finishes before the assignment to fromCharCache[c] is made. Can the compiler somehow catch these? I fear they are a small epidemic. For a long time people didn't realize where all the compiler and processor could reorder. Do we have the right constructs by now to fix them? I think we do. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 06:19:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:19:15 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> References: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Message-ID: <20120827041915.569CD1A207D@async.async.caltech.edu> If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3. The Handbook of Programming Languages has a chapter on Modula-3 as well. Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- From mika at async.caltech.edu Mon Aug 27 06:23:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:23:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: Message-ID: <20120827042351.141B81A207D@async.async.caltech.edu> Yeah it's a race condition, in theory. But is it important? If you get the wrong copy, there will be two TEXTs representing one CHAR. But no one ever said FromChar was guaranteed to return the same pointer when you call it with the same CHAR... Mika Jay K writes: >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >We have race conditions=2C like this=2C m3core/src/Text.m3: > > >VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > > >PROCEDURE FromChar (c: CHAR): T =3D > VAR buf: ARRAY [0..0] OF CHAR=3B > BEGIN > IF fromCharCache [c] =3D NIL THEN > buf [0] :=3D c=3B > fromCharCache[c] :=3D Text8.New (buf)=3B > END=3B > RETURN fromCharCache [c] > END FromChar=3B > > >It should be: > > >PROCEDURE FromChar (c: CHAR): T =3D >=0A= > VAR buf: ARRAY [0..0] OF CHAR=3B >=0A= > BEGIN >=0A= > IF fromCharCache [c] =3D NIL THEN >=0A= > buf [0] :=3D c=3B >=0A= > WITH a =3D Text8.New (buf) DO > MemoryBarrier()=3B > fromCharCache[c] :=3D a=3B > END=3B >=0A= > END=3B >=0A= > RETURN fromCharCache [c] >=0A= > END FromChar=3B >=0A= > > >to ensure that all of Text8.New() finishes before the assignment to fromCha= >rCache[c] is made. > > >Can the compiler somehow catch these? >I fear they are a small epidemic. >For a long time people didn't realize where all the compiler and processor = >could reorder. > > >Do we have the right constructs by now to fix them? >I think we do. > > > - Jay > = > >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
We have race conditions=2C like = >this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >r (c: CHAR): T =3D
=0A= > =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > =3B BEGIN
=0A= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > =3B =3B =3B END=3B
=0A= > =3B =3B =3B RETURN fromCharCache [c]
=0A= > =3B END FromChar=3B
=0A= >

to ensure that all of Text8.New() finishes before the assignment to= > fromCharCache[c] is made.


Can the compiler somehow catch these?= >
I fear they are a small epidemic.
For a long time people didn't real= >ize where all the compiler and processor could reorder.


Do we ha= >ve the right constructs by now to fix them?
I think we do.


-= > Jay
>= > >--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 06:44:21 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 04:44:21 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 07:32:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 22:32:48 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827042351.141B81A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> Message-ID: <20120827053248.A1B3F1A207D@async.async.caltech.edu> Sorry, I can't seem to quote your message. You say that Text8.New's result can be assigned to fromCharCache[c] before Text8.New is done running. How is that possible? Surely Text8.New has to calculate its result before it can be assigned anywhere. Yes possibly FromChar allocates space for the result of Text8.New but the assignment of whatever appears in that space can't happen until *after* Text8.New has run. Unless you are saying that the running of Text8.New and FromChar are intertwined somehow..? I know compilers do some odd things but that doesn't sound right... If Text8.New took fromCharCache[c] as a VAR parameter you might be right, though. But it doesn't... Am I missing something? Mika Mika Nystrom writes: > >Yeah it's a race condition, in theory. But is it important? If you get >the wrong copy, there will be two TEXTs representing one CHAR. But no >one ever said FromChar was guaranteed to return the same pointer when >you call it with the same CHAR... > > Mika > >Jay K writes: >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >>We have race conditions=2C like this=2C m3core/src/Text.m3: >> >> >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >> VAR buf: ARRAY [0..0] OF CHAR=3B >> BEGIN >> IF fromCharCache [c] =3D NIL THEN >> buf [0] :=3D c=3B >> fromCharCache[c] :=3D Text8.New (buf)=3B >> END=3B >> RETURN fromCharCache [c] >> END FromChar=3B >> >> >>It should be: >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >>=0A= >> VAR buf: ARRAY [0..0] OF CHAR=3B >>=0A= >> BEGIN >>=0A= >> IF fromCharCache [c] =3D NIL THEN >>=0A= >> buf [0] :=3D c=3B >>=0A= >> WITH a =3D Text8.New (buf) DO >> MemoryBarrier()=3B >> fromCharCache[c] :=3D a=3B >> END=3B >>=0A= >> END=3B >>=0A= >> RETURN fromCharCache [c] >>=0A= >> END FromChar=3B >>=0A= >> >> >>to ensure that all of Text8.New() finishes before the assignment to fromCha= >>rCache[c] is made. >> >> >>Can the compiler somehow catch these? >>I fear they are a small epidemic. >>For a long time people didn't realize where all the compiler and processor = >>could reorder. >> >> >>Do we have the right constructs by now to fix them? >>I think we do. >> >> >> - Jay >> = >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
We have race conditions=2C like = >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >>r (c: CHAR): T =3D
=0A= >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= >> =3B BEGIN
=0A= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= >> =3B =3B =3B END=3B
=0A= >> =3B =3B =3B RETURN fromCharCache [c]
=0A= >> =3B END FromChar=3B
=0A= >>

to ensure that all of Text8.New() finishes before the assignment to= >> fromCharCache[c] is made.


Can the compiler somehow catch these?= >>
I fear they are a small epidemic.
For a long time people didn't real= >>ize where all the compiler and processor could reorder.


Do we ha= >>ve the right constructs by now to fix them?
I think we do.


-= >> Jay
>>= >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 08:27:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 06:27:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827053248.A1B3F1A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: The processor can reorder. Across call/ret at least in general. The compiler can also reorder, if you have whole-program-optimization, link-time-code-gen (Microsoft "LTCG"), link-time-optimization (gcc "LTO"). (We don't, but real world systems certainly do.) - Jay > To: jay.krell at cornell.edu; m3devel at elegosoft.com; mika at async.caltech.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Sun, 26 Aug 2012 22:32:48 -0700 > From: mika at async.caltech.edu > > Sorry, I can't seem to quote your message. > > You say that Text8.New's result can be assigned to fromCharCache[c] > before Text8.New is done running. How is that possible? Surely > Text8.New has to calculate its result before it can be assigned > anywhere. Yes possibly FromChar allocates space for the result of > Text8.New but the assignment of whatever appears in that space can't > happen until *after* Text8.New has run. Unless you are saying that > the running of Text8.New and FromChar are intertwined somehow..? > I know compilers do some odd things but that doesn't sound right... > > If Text8.New took fromCharCache[c] as a VAR parameter you might be > right, though. But it doesn't... > > Am I missing something? > > Mika > > Mika Nystrom writes: > > > >Yeah it's a race condition, in theory. But is it important? If you get > >the wrong copy, there will be two TEXTs representing one CHAR. But no > >one ever said FromChar was guaranteed to return the same pointer when > >you call it with the same CHAR... > > > > Mika > > > >Jay K writes: > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/plain; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >>We have race conditions=2C like this=2C m3core/src/Text.m3: > >> > >> > >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >> BEGIN > >> IF fromCharCache [c] =3D NIL THEN > >> buf [0] :=3D c=3B > >> fromCharCache[c] :=3D Text8.New (buf)=3B > >> END=3B > >> RETURN fromCharCache [c] > >> END FromChar=3B > >> > >> > >>It should be: > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >>=0A= > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >>=0A= > >> BEGIN > >>=0A= > >> IF fromCharCache [c] =3D NIL THEN > >>=0A= > >> buf [0] :=3D c=3B > >>=0A= > >> WITH a =3D Text8.New (buf) DO > >> MemoryBarrier()=3B > >> fromCharCache[c] :=3D a=3B > >> END=3B > >>=0A= > >> END=3B > >>=0A= > >> RETURN fromCharCache [c] > >>=0A= > >> END FromChar=3B > >>=0A= > >> > >> > >>to ensure that all of Text8.New() finishes before the assignment to fromCha= > >>rCache[c] is made. > >> > >> > >>Can the compiler somehow catch these? > >>I fear they are a small epidemic. > >>For a long time people didn't realize where all the compiler and processor = > >>could reorder. > >> > >> > >>Do we have the right constructs by now to fix them? > >>I think we do. > >> > >> > >> - Jay > >> = > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/html; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
We have race conditions=2C like = > >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= > >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = > >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= > >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= > >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= > >>r (c: CHAR): T =3D
=0A= > >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > >> =3B BEGIN
=0A= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= > >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= > >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= > >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B RETURN fromCharCache [c]
=0A= > >> =3B END FromChar=3B
=0A= > >>

to ensure that all of Text8.New() finishes before the assignment to= > >> fromCharCache[c] is made.


Can the compiler somehow catch these?= > >>
I fear they are a small epidemic.
For a long time people didn't real= > >>ize where all the compiler and processor could reorder.


Do we ha= > >>ve the right constructs by now to fix them?
I think we do.


-= > >> Jay
> >>= > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 09:17:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 00:17:44 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: <20120827071744.781621A207D@async.async.caltech.edu> Oh I see now. It's not that the processor reorders in any tricky way. It is that the write performed by Text8.New could be sitting around processor 1's cache indefinitely without actually getting written back to main memory. Processor 2 might see the new pointer, but when reading the memory pointed to, would get uninitialized memory. I think this can happen on Alpha but maybe not X86 (without further reorderings anyhow)? Any chance any of the pthreads bugs could be of this nature? How are the pthreads doing? I gathered some bugs were fixed but I haven't tried the system for a while. Mika Jay K writes: >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >The processor can reorder. Across call/ret at least in general. > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >. (We don't=2C but real world systems certainly do.) > > > - Jay > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >> From: mika at async.caltech.edu >>=20 >> Sorry=2C I can't seem to quote your message. >>=20 >> You say that Text8.New's result can be assigned to fromCharCache[c] >> before Text8.New is done running. How is that possible? Surely >> Text8.New has to calculate its result before it can be assigned >> anywhere. Yes possibly FromChar allocates space for the result of >> Text8.New but the assignment of whatever appears in that space can't >> happen until *after* Text8.New has run. Unless you are saying that >> the running of Text8.New and FromChar are intertwined somehow..? >> I know compilers do some odd things but that doesn't sound right... >>=20 >> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> right=2C though. But it doesn't... >>=20 >> Am I missing something? >>=20 >> Mika >>=20 >> Mika Nystrom writes: >> > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= >et >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no >> >one ever said FromChar was guaranteed to return the same pointer when >> >you call it with the same CHAR... >> > >> > Mika >> > >> >Jay K writes: >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >> >> >> >> >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >exts *) >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >> BEGIN >> >> IF fromCharCache [c] =3D3D NIL THEN >> >> buf [0] :=3D3D c=3D3B >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >> >> END=3D3B >> >> RETURN fromCharCache [c] >> >> END FromChar=3D3B >> >> >> >> >> >>It should be: >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >>=3D0A=3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >>=3D0A=3D >> >> BEGIN >> >>=3D0A=3D >> >> IF fromCharCache [c] =3D3D NIL THEN >> >>=3D0A=3D >> >> buf [0] :=3D3D c=3D3B >> >>=3D0A=3D >> >> WITH a =3D3D Text8.New (buf) DO >> >> MemoryBarrier()=3D3B >> >> fromCharCache[c] :=3D3D a=3D3B >> >> END=3D3B >> >>=3D0A=3D >> >> END=3D3B >> >>=3D0A=3D >> >> RETURN fromCharCache [c] >> >>=3D0A=3D >> >> END FromChar=3D3B >> >>=3D0A=3D >> >> >> >> >> >>to ensure that all of Text8.New() finishes before the assignment to fro= >mCha=3D >> >>rCache[c] is made. >> >> >> >> >> >>Can the compiler somehow catch these? >> >>I fear they are a small epidemic. >> >>For a long time people didn't realize where all the compiler and proces= >sor =3D >> >>could reorder. >> >> >> >> >> >>Do we have the right constructs by now to fix them? >> >>I think we do. >> >> >> >> >> >> - Jay >> >> =3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >> >> >> >>
We have race conditions= >=3D2C like =3D >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > CHAR OF=3D >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >(c: CHAR=3D >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >3B BEGIN
=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >nbsp=3D3B =3D >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > =3D3B =3D3B&n=3D >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >p=3D3B =3D3B=3D >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >che [c]
&nb=3D >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > FromCha=3D >> >>r (c: CHAR): T =3D3D
=3D0A=3D >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >> >> =3D3B BEGIN
=3D0A=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >B
=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >ew (buf) DO
&=3D >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >emoryBarrier()=3D3B<=3D >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >3B fromCharCache[c=3D >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>=3D0A=3D >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >> >> =3D3B END FromChar=3D3B
=3D0A=3D >> >>

to ensure that all of Text8.New() finishes before the assignmen= >t to=3D >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= >ese?=3D >> >>
I fear they are a small epidemic.
For a long time people didn't = >real=3D >> >>ize where all the compiler and processor could reorder.


Do w= >e ha=3D >> >>ve the right constructs by now to fix them?
I think we do.

r> -=3D >> >> Jay
>> >>=3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > = > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
The processor can reorder. Acros= >s call/ret at least in general.


The compiler can also reorder=2C= > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >stems certainly do.)


 =3B- Jay


yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >is done running. How is that possible? Surely
>=3B Text8.New has to = >calculate its result before it can be assigned
>=3B anywhere. Yes pos= >sibly FromChar allocates space for the result of
>=3B Text8.New but th= >e assignment of whatever appears in that space can't
>=3B happen until= > *after* Text8.New has run. Unless you are saying that
>=3B the runni= >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >mpilers do some odd things but that doesn't sound right...
>=3B
&g= >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >one ever said FromChar was guaranteed to return the same pointer when
&g= >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >=3B >=3B>=3BFor a long time people didn't realize where all the compile= >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >-311cd8c58979_--
>= > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 10:55:09 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:55:09 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 10:58:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:58:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu>, Message-ID: Also, a compiler is free to reorder this stuff as well. If it can see the body of the function and inline it. Without an explicit lock/fence/barrier of some kind, including a call to a function that the compiler cannot see the body of, the compiler can reorder. Our compiler doesn't see far, so that helps. But that is an implementation detail of our compiler. The C backend will make "WPO" / "LTCG" / "LTO" sort of more viable..and the compiler will then see more. (all names for the same thing -- whole program optimization, link time code generation, link time optimization) - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] race conditions in lock-free code... Date: Mon, 27 Aug 2012 08:55:09 +0000 Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 14:20:56 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 13:20:56 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120827041915.569CD1A207D@async.async.caltech.edu> Message-ID: <1346070056.46764.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yet, put another "A Theory of Objects" is almost begin to end in the web, however it misses just the justice with "Baby Modula-3" as it was used to check the type system of Modula-3, I believe but this research reports never came out (or the library didn't published it) so it's difficult to know what else is missing. In fact they wrote an accompanying software to it, a new implementation of Obliq based on first chapters of the book, don't know where it ended. Thanks --- El dom, 26/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Modula-3 bitfields Para: "Daniel Alejandro Benavides D." CC: m3devel at elegosoft.com Fecha: domingo, 26 de agosto, 2012 23:19 If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3.? The Handbook of Programming Languages has a chapter on Modula-3 as well. ? ???Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Aug 27 15:27:38 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 09:27:38 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. Sent from my iPad On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The processor can reorder. Across call/ret at least in general. >> >> >> The compiler can also reorder=2C if you have whole-program-optimization=2C = >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >> . (We don't=2C but real world systems certainly do.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >> edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >>> From: mika at async.caltech.edu >>> =20 >>> Sorry=2C I can't seem to quote your message. >>> =20 >>> You say that Text8.New's result can be assigned to fromCharCache[c] >>> before Text8.New is done running. How is that possible? Surely >>> Text8.New has to calculate its result before it can be assigned >>> anywhere. Yes possibly FromChar allocates space for the result of >>> Text8.New but the assignment of whatever appears in that space can't >>> happen until *after* Text8.New has run. Unless you are saying that >>> the running of Text8.New and FromChar are intertwined somehow..? >>> I know compilers do some odd things but that doesn't sound right... >>> =20 >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>> right=2C though. But it doesn't... >>> =20 >>> Am I missing something? >>> =20 >>> Mika >>> =20 >>> Mika Nystrom writes: >>>> >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= >> et >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no >>>> one ever said FromChar was guaranteed to return the same pointer when >>>> you call it with the same CHAR... >>>> >>>> Mika >>>> >>>> Jay K writes: >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>>> >>>>> >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >> exts *) >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> BEGIN >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> buf [0] :=3D3D c=3D3B >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >>>>> END=3D3B >>>>> RETURN fromCharCache [c] >>>>> END FromChar=3D3B >>>>> >>>>> >>>>> It should be: >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> =3D0A=3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> =3D0A=3D >>>>> BEGIN >>>>> =3D0A=3D >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> =3D0A=3D >>>>> buf [0] :=3D3D c=3D3B >>>>> =3D0A=3D >>>>> WITH a =3D3D Text8.New (buf) DO >>>>> MemoryBarrier()=3D3B >>>>> fromCharCache[c] :=3D3D a=3D3B >>>>> END=3D3B >>>>> =3D0A=3D >>>>> END=3D3B >>>>> =3D0A=3D >>>>> RETURN fromCharCache [c] >>>>> =3D0A=3D >>>>> END FromChar=3D3B >>>>> =3D0A=3D >>>>> >>>>> >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= >> mCha=3D >>>>> rCache[c] is made. >>>>> >>>>> >>>>> Can the compiler somehow catch these? >>>>> I fear they are a small epidemic. >>>>> For a long time people didn't realize where all the compiler and proces= >> sor =3D >>>>> could reorder. >>>>> >>>>> >>>>> Do we have the right constructs by now to fix them? >>>>> I think we do. >>>>> >>>>> >>>>> - Jay >>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> >>>>> >>>>> >>>>>
We have race conditions= >> =3D2C like =3D >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= >> CHAR OF=3D >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >> (c: CHAR=3D >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >> 3B BEGIN
=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >> nbsp=3D3B =3D >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= >>  =3D3B =3D3B&n=3D >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >> p=3D3B =3D3B=3D >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >> che [c]
&nb=3D >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= >> FromCha=3D >>>>> r (c: CHAR): T =3D3D
=3D0A=3D >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >>>>>  =3D3B BEGIN
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >> B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >> ew (buf) DO
&=3D >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >> emoryBarrier()=3D3B<=3D >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >> 3B fromCharCache[c=3D >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D >>>>>

to ensure that all of Text8.New() finishes before the assignmen= >> t to=3D >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= >> ese?=3D >>>>>
I fear they are a small epidemic.
For a long time people didn't = >> real=3D >>>>> ize where all the compiler and processor could reorder.


Do w= >> e ha=3D >>>>> ve the right constructs by now to fix them?
I think we do.

> r> -=3D >>>>> Jay
>>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
The processor can reorder. Acros= >> s call/ret at least in general.


The compiler can also reorder=2C= >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >> stems certainly do.)


 =3B- Jay


> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >> is done running. How is that possible? Surely
>=3B Text8.New has to = >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= >> e assignment of whatever appears in that space can't
>=3B happen until= >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >> one ever said FromChar was guaranteed to return the same pointer when
&g= >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >> -311cd8c58979_--
>> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 20:07:10 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 18:07:10 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: It is NOT benign. The global can be written before the initialization of what it points to is done. The result is wrong. It IS benign if multiple threads go and allocate separate multiple initialized TEXTs. That's not the problem. - Jay > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > From: hosking at cs.purdue.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Mon, 27 Aug 2012 09:27:38 -0400 > To: mika at async.caltech.edu > > So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. > > Sent from my iPad > > On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > > > Oh I see now. > > > > It's not that the processor reorders in any tricky way. It is that the > > write performed by Text8.New could be sitting around processor 1's cache > > indefinitely without actually getting written back to main memory. > > Processor 2 might see the new pointer, but when reading the memory pointed > > to, would get uninitialized memory. I think this can happen on Alpha but > > maybe not X86 (without further reorderings anyhow)? > > > > Any chance any of the pthreads bugs could be of this nature? How are the > > pthreads doing? I gathered some bugs were fixed but I haven't tried the > > system for a while. > > > > Mika > > > > > > Jay K writes: > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> The processor can reorder. Across call/ret at least in general. > >> > >> > >> The compiler can also reorder=2C if you have whole-program-optimization=2C = > >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >> . (We don't=2C but real world systems certainly do.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >> edu > >>> Subject: Re: [M3devel] race conditions in lock-free code... > >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >>> From: mika at async.caltech.edu > >>> =20 > >>> Sorry=2C I can't seem to quote your message. > >>> =20 > >>> You say that Text8.New's result can be assigned to fromCharCache[c] > >>> before Text8.New is done running. How is that possible? Surely > >>> Text8.New has to calculate its result before it can be assigned > >>> anywhere. Yes possibly FromChar allocates space for the result of > >>> Text8.New but the assignment of whatever appears in that space can't > >>> happen until *after* Text8.New has run. Unless you are saying that > >>> the running of Text8.New and FromChar are intertwined somehow..? > >>> I know compilers do some odd things but that doesn't sound right... > >>> =20 > >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >>> right=2C though. But it doesn't... > >>> =20 > >>> Am I missing something? > >>> =20 > >>> Mika > >>> =20 > >>> Mika Nystrom writes: > >>>> > >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= > >> et > >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >>>> one ever said FromChar was guaranteed to return the same pointer when > >>>> you call it with the same CHAR... > >>>> > >>>> Mika > >>>> > >>>> Jay K writes: > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >>>>> > >>>>> > >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >> exts *) > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> BEGIN > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> buf [0] :=3D3D c=3D3B > >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >>>>> END=3D3B > >>>>> RETURN fromCharCache [c] > >>>>> END FromChar=3D3B > >>>>> > >>>>> > >>>>> It should be: > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> =3D0A=3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> =3D0A=3D > >>>>> BEGIN > >>>>> =3D0A=3D > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> =3D0A=3D > >>>>> buf [0] :=3D3D c=3D3B > >>>>> =3D0A=3D > >>>>> WITH a =3D3D Text8.New (buf) DO > >>>>> MemoryBarrier()=3D3B > >>>>> fromCharCache[c] :=3D3D a=3D3B > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> RETURN fromCharCache [c] > >>>>> =3D0A=3D > >>>>> END FromChar=3D3B > >>>>> =3D0A=3D > >>>>> > >>>>> > >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= > >> mCha=3D > >>>>> rCache[c] is made. > >>>>> > >>>>> > >>>>> Can the compiler somehow catch these? > >>>>> I fear they are a small epidemic. > >>>>> For a long time people didn't realize where all the compiler and proces= > >> sor =3D > >>>>> could reorder. > >>>>> > >>>>> > >>>>> Do we have the right constructs by now to fix them? > >>>>> I think we do. > >>>>> > >>>>> > >>>>> - Jay > >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>
We have race conditions= > >> =3D2C like =3D > >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > >> CHAR OF=3D > >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >> (c: CHAR=3D > >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >> 3B BEGIN
=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >> nbsp=3D3B =3D > >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > >>  =3D3B =3D3B&n=3D > >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >> p=3D3B =3D3B=3D > >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >> che [c]
&nb=3D > >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > >> FromCha=3D > >>>>> r (c: CHAR): T =3D3D
=3D0A=3D > >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >>>>>  =3D3B BEGIN
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >> B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >> ew (buf) DO
&=3D > >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >> emoryBarrier()=3D3B<=3D > >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >> 3B fromCharCache[c=3D > >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D > >>>>>

to ensure that all of Text8.New() finishes before the assignmen= > >> t to=3D > >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= > >> ese?=3D > >>>>>
I fear they are a small epidemic.
For a long time people didn't = > >> real=3D > >>>>> ize where all the compiler and processor could reorder.


Do w= > >> e ha=3D > >>>>> ve the right constructs by now to fix them?
I think we do.

>> r> -=3D > >>>>> Jay
> >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- > >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
The processor can reorder. Acros= > >> s call/ret at least in general.


The compiler can also reorder=2C= > >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >> stems certainly do.)


 =3B- Jay


>> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >> is done running. How is that possible? Surely
>=3B Text8.New has to = > >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >> e assignment of whatever appears in that space can't
>=3B happen until= > >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >> one ever said FromChar was guaranteed to return the same pointer when
&g= > >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= > >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >> -311cd8c58979_--
> >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 20:34:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 11:34:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: <20120827183451.976971A2094@async.async.caltech.edu> yeah I agree. processor 1 performs... x := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent. If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive. In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2. (Or Exclusive, if P1 writes it back and invalidates.) This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global. The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice. I'd suspect that a multiprocessor 21264 could be. But I do agree it's a bug relative to the Alpha Architecture Manual. A very nasty one at that. Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily. The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way. It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory. I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature? How are t= >he >> > pthreads doing? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> > Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running. How is that possible? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere. Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run. Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though. But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>> Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory. But is it important? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>> Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>> END=3D3D3B >> >>>>> RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> WITH a =3D3D3D Text8.New (buf) DO >> >>>>> MemoryBarrier()=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running. How is that possible? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere. = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run. Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >t=3D3B IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >B RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B =3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B =3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily. The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way. It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere. Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory. But is it important? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= > buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = > <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B =3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From dabenavidesd at yahoo.es Mon Aug 27 23:54:16 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 22:54:16 +0100 (BST) Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <1346104456.75314.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I believe there shouldn't be so many for not being counted with fingers of one hand. The memory model was specified and model checked with TLA by Lamport and publicly available for bug finders contest. However being an UNSAFE MODULE I suspect this was intended to explode that relaxed restriction. Also probable a C weak semantics could make things worse or better than they are, so ... Thanks in advance --- El lun, 27/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] race conditions in lock-free code... Para: "Jay K" CC: m3devel at elegosoft.com Fecha: lunes, 27 de agosto, 2012 13:34 yeah I agree. processor 1 performs... x? := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent.? If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive.? In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2.? (Or Exclusive, if P1 writes it back and invalidates.)? This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global.? The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice.? I'd suspect that a multiprocessor 21264 could be.? But I do agree it's a bug relative to the Alpha Architecture Manual.? A very nasty one at that. ? ? Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily.? The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way.? It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory.? I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature?? How are t= >he >> > pthreads doing?? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> >? ???Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running.? How is that possible?? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere.? Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run.? Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though.? But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>>? ???Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory.? But is it important?? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR.? = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>>? ? Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>>? ???fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>>???END=3D3D3B >> >>>>>???RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>? ???WITH a =3D3D3D Text8.New (buf) DO >> >>>>>? ? ???MemoryBarrier()=3D3D3B >> >>>>>? ? ???fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>>? ? END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>>? ? ? ? ? ? ? ? ? ? ? ???=3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
? ? ? ? ? ? ? ? ? ? ? ???
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >>? ? ? ? ? ? ? ? ? ? ? ???=3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running.? How is that possible?? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere.? = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run.? Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though.? But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B? ? ? Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory.? But is it important?? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR.? But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B? ???Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B? VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B&g= >t=3D3B? ? IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B? ? ? buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B? ? ? fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? END=3D3D3B
>=3D3B >=3D3B>=3D3= >B? ? RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B? END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B? VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B???=3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? ? WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B? ? ? ? MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B? ? ? ? fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B? ???END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
? ? ? ? ? ? ? ? ? ? ? ???
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > ??? ???????? ?????? ??? ? = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily.? The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way.? It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory.? I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature?? How are the
>=3B >=3B pthreads do= >ing?? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B? ???Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible?? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere.? Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run.? Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though.? But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B? ???Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory.? But is it important?? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR.? But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B? ? Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B? ???buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B? ???fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B???END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >? ???buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B? ???WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B???END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B? ? ? ? ? ? ? ? ? ? ???= >? <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning.? How is that possible?? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere.? Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run.? Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though.? But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B? ? ? Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory.? But is it important?? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR.? But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B? ???Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B? VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ???= > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B? BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B???=3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B? ? ? ? MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? ? fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B? ???END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B? ? ? ? ? ? ? ? ? ? ? ???<=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
??? ???????? ?????? ??? ? v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 28 02:40:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 20:40:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > > yeah I agree. > > processor 1 performs... > > x := alloc_new_text > x^ := character > return x > global := x > > processor 2 performs > > IF global # NIL THEN use(global) END > > now if out of all this, x^ := character is held back, e.g., in a processor > write buffer, and if use(global) involves performing global^, the value > of global^ will be the value of global^ just after x := alloc_new_text > and before x^ := character. > > I believe very aggressive architectures such as Alpha permit reordering > to that extent. If memory serves, on Alpha, if you do not perform a > Memory Barrier, pretty much nothing can be inferred on one processor > about the ordering of operations on another. > > X86 I am not sure is that aggressive. In a hardware-synchronized > multiprocessor environment without barrier operations using e.g. the > Illinois/MESI protocol (doesn't X86?), at the point x^ := character is > performed, the cache coherence protocol has to guarantee the cache line > is in state Exclusive, which means it has to be evicted from the cache > in processor 2 and when use(global) comes around the protocol will > have to ensure it is in Shared state on processor 2. (Or Exclusive, > if P1 writes it back and invalidates.) This would guarantee that you > could not actually see a very old value for global^ at the same time > as a newer value for global. The protocol guarantees that the relative > ordering of writes is maintained across processors. > > Of course I don't know whether Alphas are actually that aggressive in > practice. I'd suspect that a multiprocessor 21264 could be. But I do > agree it's a bug relative to the Alpha Architecture Manual. A very nasty > one at that. > > Mika > > > Jay K writes: >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> It is NOT benign. >> The global can be written before the initialization of what it points to is= >> done. >> The result is wrong. >> >> >> It IS benign if multiple threads go and allocate separate multiple initiali= >> zed TEXTs. >> That's not the problem. >> >> >> - Jay >> >> >>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>> From: hosking at cs.purdue.edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>> To: mika at async.caltech.edu >>> =20 >>> So why is this anything other than benign? It's a cache and there's no pr= >> oblem if different threads get different texts. Don't cripple performance u= >> nnecessarily. The collector will reclaim any unreachable text. >>> =20 >>> Sent from my iPad >>> =20 >>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >> wrote: >>> =20 >>>> Oh I see now. >>>> =20 >>>> It's not that the processor reorders in any tricky way. It is that the >>>> write performed by Text8.New could be sitting around processor 1's cach= >> e >>>> indefinitely without actually getting written back to main memory. >>>> Processor 2 might see the new pointer=2C but when reading the memory po= >> inted >>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >> but >>>> maybe not X86 (without further reorderings anyhow)? >>>> =20 >>>> Any chance any of the pthreads bugs could be of this nature? How are t= >> he >>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >> e >>>> system for a while. >>>> =20 >>>> Mika >>>> =20 >>>> =20 >>>> Jay K writes: >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> The processor can reorder. Across call/ret at least in general. >>>>> =20 >>>>> =20 >>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >> ion=3D2C =3D >>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >> "LTO")=3D >>>>> . (We don't=3D2C but real world systems certainly do.) >>>>> =20 >>>>> =20 >>>>> - Jay >>>>> =20 >>>>> =20 >>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >> caltech.=3D >>>>> edu >>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> =3D20 >>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>> =3D20 >>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>> before Text8.New is done running. How is that possible? Surely >>>>>> Text8.New has to calculate its result before it can be assigned >>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>> =3D20 >>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>> right=3D2C though. But it doesn't... >>>>>> =3D20 >>>>>> Am I missing something? >>>>>> =3D20 >>>>>> Mika >>>>>> =3D20 >>>>>> Mika Nystrom writes: >>>>>>> =20 >>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >> you g=3D >>>>> et >>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >> But no >>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >> n >>>>>>> you call it with the same CHAR... >>>>>>> =20 >>>>>>> Mika >>>>>>> =20 >>>>>>> Jay K writes: >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >> * 1-char t=3D >>>>> exts *) >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> BEGIN >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> RETURN fromCharCache [c] >>>>>>>> END FromChar=3D3D3B >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> It should be: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> =3D3D0A=3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> BEGIN >>>>>>>> =3D3D0A=3D3D >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> =3D3D0A=3D3D >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> RETURN fromCharCache [c] >>>>>>>> =3D3D0A=3D3D >>>>>>>> END FromChar=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >> fro=3D >>>>> mCha=3D3D >>>>>>>> rCache[c] is made. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Can the compiler somehow catch these? >>>>>>>> I fear they are a small epidemic. >>>>>>>> For a long time people didn't realize where all the compiler and pr= >> oces=3D >>>>> sor =3D3D >>>>>>>> could reorder. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>> I think we do. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> - Jay >>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>
We have race con= >> ditions=3D >>>>> =3D3D2C like =3D3D >>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >> 3D ARRAY=3D >>>>> CHAR OF=3D3D >>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >> omChar =3D >>>>> (c: CHAR=3D3D >>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B> r> =3D3D=3D >>>>> 3B BEGIN
=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
&=3D >>>>> nbsp=3D3D3B =3D3D >>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >> 3B
 =3D3D3B=3D >>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >> 3D3B
&nbs=3D >>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >> URN fromCharCa=3D >>>>> che [c]
&nb=3D3D >>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >> ROCEDURE=3D >>>>> FromCha=3D3D >>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
=3D >>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >> ] :=3D3D3D c=3D3D3=3D >>>>> B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >> =3D3D3D Text8.N=3D >>>>> ew (buf) DO
&=3D3D >>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B =3D3D3B M=3D >>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >> p=3D3D3B =3D3D=3D >>>>> 3B fromCharCache[c=3D3D >>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B END=3D3D3B>>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >> =3D3D0A=3D3D >>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >> nmen=3D >>>>> t to=3D3D >>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >> th=3D >>>>> ese?=3D3D >>>>>>>>
I fear they are a small epidemic.
For a long time people did= >> n't =3D >>>>> real=3D3D >>>>>>>> ize where all the compiler and processor could reorder.


= >> Do w=3D >>>>> e ha=3D3D >>>>>>>> ve the right constructs by now to fix them?
I think we do.
> r>>>>> r> -=3D3D >>>>>>>> Jay
>>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> >>>>> >>>>> >>>>>
The processor can reord= >> er. Acros=3D >>>>> s call/ret at least in general.


The compiler can also reord= >> er=3D2C=3D >>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >> oft "L=3D >>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >> world sy=3D >>>>> stems certainly do.)


 =3D3B- Jay


> d=3D3D"Sk=3D >>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >> el at elegos=3D >>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >> race con=3D >>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >> :32:48 -0=3D >>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >> Sorry=3D2C I =3D >>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >> t Text8.N=3D >>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >> t8.New =3D >>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >> has to =3D >>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >> Yes pos=3D >>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >> but th=3D >>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >> n until=3D >>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >> runni=3D >>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >> know co=3D >>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>
&g=3D >>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >> be
=3D >>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >> B Am I missin=3D >>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >> 3B Mika Nystr=3D >>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >> ondition=3D2C i=3D >>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >> ong copy=3D >>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >> 3B >=3D3B=3D >>>>> one ever said FromChar was guaranteed to return the same pointer when<= >> br>&g=3D >>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >> br>>=3D3B >=3D >>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >> =3D3B>=3D3BCon=3D >>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >> 3B>=3D3BConte=3D >>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>> >=3D3B >=3D >>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >> c/Text.m3:>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >> t=3D3B>=3D3BVAR fromC=3D >>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >> ts *)
&g=3D >>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3BPROCEDURE Fr=3D >>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >> AY [0..0] OF =3D >>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >> t=3D3B IF fromCh=3D >>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >> :=3D3D3D c=3D3D3=3D >>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >> w (buf)=3D3D3B<=3D >>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >> B RETURN fromCha=3D >>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >> 3B >=3D3B>=3D3B=3D >>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >> :
>=3D3B >=3D >>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >> BPROCEDURE FromChar (=3D >>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B VA=3D >>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >> =3D3D
>=3D3B &=3D >>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >> =3D3B >=3D3B>=3D3B =3D >>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >> D0A=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>> >=3D3B >=3D >>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >> 3B fromCha=3D >>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >> B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B> r>>=3D3B >=3D3B>=3D3B=3D >>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >> =3D3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B
>=3D3B &=3D >>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >> ssignment=3D >>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >> =3D3B >=3D3B>=3D >>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >> compiler somehow c=3D >>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >> ic.
>=3D >>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >> the compile=3D >>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >> t=3D3B >=3D3B&g=3D >>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >> have the right cons=3D >>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >> br>>=3D3B &=3D >>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >> =3D3B - Jay
>=3D3B &=3D >>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >> ent-Type: t=3D >>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >> tent-Transfer-E=3D >>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D >>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >> >=3D3B >=3D3B>=3D3B=3D >>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >> e P
>=3D3B >=3D >>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >> 3B >=3D3B>=3D3Bpadd=3D >>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >> .hmmessage
>=3D >>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >> 3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >> =3D3B >=3D3B>=3D3B=3D >>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D3Bbod=3D >>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >> have race cond=3D >>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >> e/src/Text.m3:<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >> =3D3D3D ARRAY CHAR =3D >>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >> AR=3D3D
>=3D3B=3D >>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >> buf: ARRAY [0..0]=3D >>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >> t=3D3B=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >> Bnbsp=3D3D3B IF fromCha=3D >>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >> =3D3Bnbsp=3D3D
&g=3D >>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B buf =3D >>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >> sp=3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B fromCharCac=3D >>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >> 3B<=3D3Bbr>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >> arCache [c]<=3D3B=3D >>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >> romChar=3D3D3B<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >> t=3D3B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >> =3D3Br (c: CHAR): T =3D >>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >> >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >> =3D3D3D NIL THEN&l=3D >>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >> =3D3D3D c=3D3D3B<=3D >>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >> 3D3D Text8.New (buf=3D >>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B&=3D3Bnbs=3D >>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B&am=3D >>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >> >=3D3B>=3D3B] :=3D >>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >> amp=3D3Bnbsp=3D3D3B RET=3D >>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >> 3B>=3D3B&=3D3Bnb=3D >>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >> B >=3D3B>=3D3B<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >> ishes before =3D >>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >> s made.<=3D3B=3D >>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >> catch these?=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >> small epidemic.&l=3D >>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >> t=3D3B>=3D3Bize=3D >>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >> lt=3D3Bbr&=3D >>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >> the right construct=3D >>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >> B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >> Bbr>=3D3B =3D >>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >> <=3D3B/html>=3D3B=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >> 896e-68ee-44da-82dd=3D >>>>> -311cd8c58979_--
>>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
It is NOT benign.
The global = >> can be written before the initialization of what it points to is done.
T= >> he result is wrong.


It IS benign if multiple threads go and allo= >> cate separate multiple initialized TEXTs.
That's not the problem.
>>
 =3B- Jay


&g= >> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >> r than benign? It's a cache and there's no problem if different threads get= >> different texts. Don't cripple performance unnecessarily. The collector w= >> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >> ly without actually getting written back to main memory.
>=3B >=3B P= >> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >> ensure that all of Text8.New() finishes before the assignment to fro=3D>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >> long time people didn't realize where all the compiler and proces=3D
&g= >> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
> v> >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From jay.krell at cornell.edu Tue Aug 28 04:45:45 2012 From: jay.krell at cornell.edu (Jay) Date: Mon, 27 Aug 2012 19:45:45 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. > > Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. > > It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile +1 765 427 5484 > > > > > > On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > >> >> yeah I agree. >> >> processor 1 performs... >> >> x := alloc_new_text >> x^ := character >> return x >> global := x >> >> processor 2 performs >> >> IF global # NIL THEN use(global) END >> >> now if out of all this, x^ := character is held back, e.g., in a processor >> write buffer, and if use(global) involves performing global^, the value >> of global^ will be the value of global^ just after x := alloc_new_text >> and before x^ := character. >> >> I believe very aggressive architectures such as Alpha permit reordering >> to that extent. If memory serves, on Alpha, if you do not perform a >> Memory Barrier, pretty much nothing can be inferred on one processor >> about the ordering of operations on another. >> >> X86 I am not sure is that aggressive. In a hardware-synchronized >> multiprocessor environment without barrier operations using e.g. the >> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >> performed, the cache coherence protocol has to guarantee the cache line >> is in state Exclusive, which means it has to be evicted from the cache >> in processor 2 and when use(global) comes around the protocol will >> have to ensure it is in Shared state on processor 2. (Or Exclusive, >> if P1 writes it back and invalidates.) This would guarantee that you >> could not actually see a very old value for global^ at the same time >> as a newer value for global. The protocol guarantees that the relative >> ordering of writes is maintained across processors. >> >> Of course I don't know whether Alphas are actually that aggressive in >> practice. I'd suspect that a multiprocessor 21264 could be. But I do >> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >> one at that. >> >> Mika >> >> >> Jay K writes: >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> It is NOT benign. >>> The global can be written before the initialization of what it points to is= >>> done. >>> The result is wrong. >>> >>> >>> It IS benign if multiple threads go and allocate separate multiple initiali= >>> zed TEXTs. >>> That's not the problem. >>> >>> >>> - Jay >>> >>> >>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>> From: hosking at cs.purdue.edu >>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>> To: mika at async.caltech.edu >>>> =20 >>>> So why is this anything other than benign? It's a cache and there's no pr= >>> oblem if different threads get different texts. Don't cripple performance u= >>> nnecessarily. The collector will reclaim any unreachable text. >>>> =20 >>>> Sent from my iPad >>>> =20 >>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>> wrote: >>>> =20 >>>>> Oh I see now. >>>>> =20 >>>>> It's not that the processor reorders in any tricky way. It is that the >>>>> write performed by Text8.New could be sitting around processor 1's cach= >>> e >>>>> indefinitely without actually getting written back to main memory. >>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>> inted >>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>> but >>>>> maybe not X86 (without further reorderings anyhow)? >>>>> =20 >>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>> he >>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>> e >>>>> system for a while. >>>>> =20 >>>>> Mika >>>>> =20 >>>>> =20 >>>>> Jay K writes: >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> The processor can reorder. Across call/ret at least in general. >>>>>> =20 >>>>>> =20 >>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>> ion=3D2C =3D >>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>> "LTO")=3D >>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>> =20 >>>>>> =20 >>>>>> - Jay >>>>>> =20 >>>>>> =20 >>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>> caltech.=3D >>>>>> edu >>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>> From: mika at async.caltech.edu >>>>>>> =3D20 >>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>> =3D20 >>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>> =3D20 >>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>> right=3D2C though. But it doesn't... >>>>>>> =3D20 >>>>>>> Am I missing something? >>>>>>> =3D20 >>>>>>> Mika >>>>>>> =3D20 >>>>>>> Mika Nystrom writes: >>>>>>>> =20 >>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>> you g=3D >>>>>> et >>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>> But no >>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>> n >>>>>>>> you call it with the same CHAR... >>>>>>>> =20 >>>>>>>> Mika >>>>>>>> =20 >>>>>>>> Jay K writes: >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>> * 1-char t=3D >>>>>> exts *) >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> BEGIN >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> It should be: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> BEGIN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>> fro=3D >>>>>> mCha=3D3D >>>>>>>>> rCache[c] is made. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Can the compiler somehow catch these? >>>>>>>>> I fear they are a small epidemic. >>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>> oces=3D >>>>>> sor =3D3D >>>>>>>>> could reorder. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>> I think we do. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> - Jay >>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>
We have race con= >>> ditions=3D >>>>>> =3D3D2C like =3D3D >>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>> 3D ARRAY=3D >>>>>> CHAR OF=3D3D >>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>> omChar =3D >>>>>> (c: CHAR=3D3D >>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>> r> =3D3D=3D >>>>>> 3B BEGIN
=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
&=3D >>>>>> nbsp=3D3D3B =3D3D >>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>> 3B
 =3D3D3B=3D >>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>> 3D3B
&nbs=3D >>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>> URN fromCharCa=3D >>>>>> che [c]
&nb=3D3D >>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>> ROCEDURE=3D >>>>>> FromCha=3D3D >>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
=3D >>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>> ] :=3D3D3D c=3D3D3=3D >>>>>> B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>> =3D3D3D Text8.N=3D >>>>>> ew (buf) DO
&=3D3D >>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B =3D3D3B M=3D >>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>> p=3D3D3B =3D3D=3D >>>>>> 3B fromCharCache[c=3D3D >>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B END=3D3D3B>>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>> nmen=3D >>>>>> t to=3D3D >>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>> th=3D >>>>>> ese?=3D3D >>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>> n't =3D >>>>>> real=3D3D >>>>>>>>> ize where all the compiler and processor could reorder.


= >>> Do w=3D >>>>>> e ha=3D3D >>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>> r>>>>>> r> -=3D3D >>>>>>>>> Jay
>>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> >>>>>> >>>>>> >>>>>>
The processor can reord= >>> er. Acros=3D >>>>>> s call/ret at least in general.


The compiler can also reord= >>> er=3D2C=3D >>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>> oft "L=3D >>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>> world sy=3D >>>>>> stems certainly do.)


 =3D3B- Jay


>> d=3D3D"Sk=3D >>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>> el at elegos=3D >>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>> race con=3D >>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>> :32:48 -0=3D >>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>> Sorry=3D2C I =3D >>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>> t Text8.N=3D >>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>> t8.New =3D >>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>> has to =3D >>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>> Yes pos=3D >>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>> but th=3D >>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>> n until=3D >>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>> runni=3D >>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>> know co=3D >>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>
&g=3D >>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>> be
=3D >>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>> B Am I missin=3D >>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>> 3B Mika Nystr=3D >>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>> ondition=3D2C i=3D >>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>> ong copy=3D >>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>> 3B >=3D3B=3D >>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>> br>&g=3D >>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>> br>>=3D3B >=3D >>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>> =3D3B>=3D3BCon=3D >>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>> 3B>=3D3BConte=3D >>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>> >=3D3B >=3D >>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>> c/Text.m3:>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>> t=3D3B>=3D3BVAR fromC=3D >>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>> ts *)
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>> AY [0..0] OF =3D >>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>> t=3D3B IF fromCh=3D >>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>> :=3D3D3D c=3D3D3=3D >>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>> w (buf)=3D3D3B<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>> B RETURN fromCha=3D >>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3B=3D >>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>> :
>=3D3B >=3D >>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>> BPROCEDURE FromChar (=3D >>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B VA=3D >>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>> =3D3D
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>> =3D3B >=3D3B>=3D3B =3D >>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>> D0A=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>> >=3D3B >=3D >>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>> 3B fromCha=3D >>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>> B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>> =3D3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B
>=3D3B &=3D >>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>> ssignment=3D >>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>> =3D3B >=3D3B>=3D >>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>> compiler somehow c=3D >>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>> ic.
>=3D >>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>> the compile=3D >>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>> t=3D3B >=3D3B&g=3D >>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>> have the right cons=3D >>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>> br>>=3D3B &=3D >>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>> =3D3B - Jay
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>> ent-Type: t=3D >>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>> tent-Transfer-E=3D >>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D >>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>> >=3D3B >=3D3B>=3D3B=3D >>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>> e P
>=3D3B >=3D >>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3Bpadd=3D >>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>> .hmmessage
>=3D >>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>> 3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>> =3D3B >=3D3B>=3D3B=3D >>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>> have race cond=3D >>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>> e/src/Text.m3:<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>> =3D3D3D ARRAY CHAR =3D >>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>> AR=3D3D
>=3D3B=3D >>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>> buf: ARRAY [0..0]=3D >>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>> t=3D3B=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>> Bnbsp=3D3D3B IF fromCha=3D >>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>> =3D3Bnbsp=3D3D
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B fromCharCac=3D >>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>> 3B<=3D3Bbr>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>> arCache [c]<=3D3B=3D >>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>> romChar=3D3D3B<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>> t=3D3B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>> =3D3Br (c: CHAR): T =3D >>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>> >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>> =3D3D3D NIL THEN&l=3D >>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>> =3D3D3D c=3D3D3B<=3D >>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>> 3D3D Text8.New (buf=3D >>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>> >=3D3B>=3D3B] :=3D >>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>> 3B>=3D3B&=3D3Bnb=3D >>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>> B >=3D3B>=3D3B<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>> ishes before =3D >>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>> s made.<=3D3B=3D >>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>> catch these?=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>> small epidemic.&l=3D >>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>> t=3D3B>=3D3Bize=3D >>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>> lt=3D3Bbr&=3D >>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>> the right construct=3D >>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>> B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>> Bbr>=3D3B =3D >>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>> <=3D3B/html>=3D3B=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>> 896e-68ee-44da-82dd=3D >>>>>> -311cd8c58979_--
>>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/html; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> >>> >>> >>>
It is NOT benign.
The global = >>> can be written before the initialization of what it points to is done.
T= >>> he result is wrong.


It IS benign if multiple threads go and allo= >>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>
 =3B- Jay


&g= >>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>> r than benign? It's a cache and there's no problem if different threads get= >>> different texts. Don't cripple performance unnecessarily. The collector w= >>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>> ly without actually getting written back to main memory.
>=3B >=3B P= >>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>> long time people didn't realize where all the compiler and proces=3D
&g= >>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>> v> >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- > From hosking at cs.purdue.edu Tue Aug 28 05:21:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 23:21:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: <80443A80-9E57-4994-B0EF-B31C5FC98EB5@cs.purdue.edu> Indeed, a lock is overkill. In general any constructor (initializer) will need a StoreStore barrier (see http://g.oswego.edu/dl/jmm/cookbook.html) before returning a pointer to the constructed storage. Sent from my iPad On Aug 27, 2012, at 10:45 PM, Jay wrote: > Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. SPARC is TSO. So not necessary. StoreStore = nop. ARM needs dmb-st. Alpha needs wmb. I don't know what a StoreStore barrier looks like on MIPS. > > A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. > > Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... Overkill. Don't scare the horses unnecessarily. Concurrent programmers must always beware. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > >> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. >> >> Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. >> >> It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. >> >> Antony Hosking | Associate Professor | Computer Science | Purdue University >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Mobile +1 765 427 5484 >> >> >> >> >> >> On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: >> >>> >>> yeah I agree. >>> >>> processor 1 performs... >>> >>> x := alloc_new_text >>> x^ := character >>> return x >>> global := x >>> >>> processor 2 performs >>> >>> IF global # NIL THEN use(global) END >>> >>> now if out of all this, x^ := character is held back, e.g., in a processor >>> write buffer, and if use(global) involves performing global^, the value >>> of global^ will be the value of global^ just after x := alloc_new_text >>> and before x^ := character. >>> >>> I believe very aggressive architectures such as Alpha permit reordering >>> to that extent. If memory serves, on Alpha, if you do not perform a >>> Memory Barrier, pretty much nothing can be inferred on one processor >>> about the ordering of operations on another. >>> >>> X86 I am not sure is that aggressive. In a hardware-synchronized >>> multiprocessor environment without barrier operations using e.g. the >>> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >>> performed, the cache coherence protocol has to guarantee the cache line >>> is in state Exclusive, which means it has to be evicted from the cache >>> in processor 2 and when use(global) comes around the protocol will >>> have to ensure it is in Shared state on processor 2. (Or Exclusive, >>> if P1 writes it back and invalidates.) This would guarantee that you >>> could not actually see a very old value for global^ at the same time >>> as a newer value for global. The protocol guarantees that the relative >>> ordering of writes is maintained across processors. >>> >>> Of course I don't know whether Alphas are actually that aggressive in >>> practice. I'd suspect that a multiprocessor 21264 could be. But I do >>> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >>> one at that. >>> >>> Mika >>> >>> >>> Jay K writes: >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> It is NOT benign. >>>> The global can be written before the initialization of what it points to is= >>>> done. >>>> The result is wrong. >>>> >>>> >>>> It IS benign if multiple threads go and allocate separate multiple initiali= >>>> zed TEXTs. >>>> That's not the problem. >>>> >>>> >>>> - Jay >>>> >>>> >>>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>>> From: hosking at cs.purdue.edu >>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>>> To: mika at async.caltech.edu >>>>> =20 >>>>> So why is this anything other than benign? It's a cache and there's no pr= >>>> oblem if different threads get different texts. Don't cripple performance u= >>>> nnecessarily. The collector will reclaim any unreachable text. >>>>> =20 >>>>> Sent from my iPad >>>>> =20 >>>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>>> wrote: >>>>> =20 >>>>>> Oh I see now. >>>>>> =20 >>>>>> It's not that the processor reorders in any tricky way. It is that the >>>>>> write performed by Text8.New could be sitting around processor 1's cach= >>>> e >>>>>> indefinitely without actually getting written back to main memory. >>>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>>> inted >>>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>>> but >>>>>> maybe not X86 (without further reorderings anyhow)? >>>>>> =20 >>>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>>> he >>>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>>> e >>>>>> system for a while. >>>>>> =20 >>>>>> Mika >>>>>> =20 >>>>>> =20 >>>>>> Jay K writes: >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> The processor can reorder. Across call/ret at least in general. >>>>>>> =20 >>>>>>> =20 >>>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>>> ion=3D2C =3D >>>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>>> "LTO")=3D >>>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>>> =20 >>>>>>> =20 >>>>>>> - Jay >>>>>>> =20 >>>>>>> =20 >>>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>>> caltech.=3D >>>>>>> edu >>>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>>> From: mika at async.caltech.edu >>>>>>>> =3D20 >>>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>>> =3D20 >>>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>>> =3D20 >>>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>>> right=3D2C though. But it doesn't... >>>>>>>> =3D20 >>>>>>>> Am I missing something? >>>>>>>> =3D20 >>>>>>>> Mika >>>>>>>> =3D20 >>>>>>>> Mika Nystrom writes: >>>>>>>>> =20 >>>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>>> you g=3D >>>>>>> et >>>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>>> But no >>>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>>> n >>>>>>>>> you call it with the same CHAR... >>>>>>>>> =20 >>>>>>>>> Mika >>>>>>>>> =20 >>>>>>>>> Jay K writes: >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>>> * 1-char t=3D >>>>>>> exts *) >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> BEGIN >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> It should be: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> BEGIN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>>> fro=3D >>>>>>> mCha=3D3D >>>>>>>>>> rCache[c] is made. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Can the compiler somehow catch these? >>>>>>>>>> I fear they are a small epidemic. >>>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>>> oces=3D >>>>>>> sor =3D3D >>>>>>>>>> could reorder. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>>> I think we do. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> - Jay >>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>
We have race con= >>>> ditions=3D >>>>>>> =3D3D2C like =3D3D >>>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>>> 3D ARRAY=3D >>>>>>> CHAR OF=3D3D >>>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>>> omChar =3D >>>>>>> (c: CHAR=3D3D >>>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>>> r> =3D3D=3D >>>>>>> 3B BEGIN
=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
&=3D >>>>>>> nbsp=3D3D3B =3D3D >>>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>>> 3B
 =3D3D3B=3D >>>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>>> 3D3B
&nbs=3D >>>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>>> URN fromCharCa=3D >>>>>>> che [c]
&nb=3D3D >>>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>>> ROCEDURE=3D >>>>>>> FromCha=3D3D >>>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
=3D >>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>>> ] :=3D3D3D c=3D3D3=3D >>>>>>> B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>>> =3D3D3D Text8.N=3D >>>>>>> ew (buf) DO
&=3D3D >>>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B =3D3D3B M=3D >>>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>>> p=3D3D3B =3D3D=3D >>>>>>> 3B fromCharCache[c=3D3D >>>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B END=3D3D3B>>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>>> nmen=3D >>>>>>> t to=3D3D >>>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>>> th=3D >>>>>>> ese?=3D3D >>>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>>> n't =3D >>>>>>> real=3D3D >>>>>>>>>> ize where all the compiler and processor could reorder.


= >>>> Do w=3D >>>>>>> e ha=3D3D >>>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>>> r>>>>>>> r> -=3D3D >>>>>>>>>> Jay
>>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> >>>>>>> >>>>>>> >>>>>>>
The processor can reord= >>>> er. Acros=3D >>>>>>> s call/ret at least in general.


The compiler can also reord= >>>> er=3D2C=3D >>>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>>> oft "L=3D >>>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>>> world sy=3D >>>>>>> stems certainly do.)


 =3D3B- Jay


>>> d=3D3D"Sk=3D >>>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>>> el at elegos=3D >>>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>>> race con=3D >>>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>>> :32:48 -0=3D >>>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>>> Sorry=3D2C I =3D >>>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>>> t Text8.N=3D >>>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>>> t8.New =3D >>>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>>> has to =3D >>>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>>> Yes pos=3D >>>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>>> but th=3D >>>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>>> n until=3D >>>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>>> runni=3D >>>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>>> know co=3D >>>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>>
&g=3D >>>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>>> be
=3D >>>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>>> B Am I missin=3D >>>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>>> 3B Mika Nystr=3D >>>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>>> ondition=3D2C i=3D >>>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>>> ong copy=3D >>>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>>> 3B >=3D3B=3D >>>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>>> br>&g=3D >>>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>>> =3D3B>=3D3BCon=3D >>>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>>> 3B>=3D3BConte=3D >>>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>>> c/Text.m3:>>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>>> t=3D3B>=3D3BVAR fromC=3D >>>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>>> ts *)
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>>> AY [0..0] OF =3D >>>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>>> t=3D3B IF fromCh=3D >>>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>>> :=3D3D3D c=3D3D3=3D >>>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>>> w (buf)=3D3D3B<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>>> B RETURN fromCha=3D >>>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3B=3D >>>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>>> :
>=3D3B >=3D >>>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>>> BPROCEDURE FromChar (=3D >>>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B VA=3D >>>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>>> =3D3D
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>>> =3D3B >=3D3B>=3D3B =3D >>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>>> D0A=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>>> 3B fromCha=3D >>>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>>> B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>>> =3D3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>>> ssignment=3D >>>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>>> =3D3B >=3D3B>=3D >>>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>>> compiler somehow c=3D >>>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>>> ic.
>=3D >>>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>>> the compile=3D >>>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>>> t=3D3B >=3D3B&g=3D >>>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>>> have the right cons=3D >>>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>>> br>>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>>> =3D3B - Jay
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>>> ent-Type: t=3D >>>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>>> tent-Transfer-E=3D >>>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D >>>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>>> >=3D3B >=3D3B>=3D3B=3D >>>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>>> e P
>=3D3B >=3D >>>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3Bpadd=3D >>>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>>> .hmmessage
>=3D >>>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>>> 3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>>> =3D3B >=3D3B>=3D3B=3D >>>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>>> have race cond=3D >>>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>>> e/src/Text.m3:<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>>> =3D3D3D ARRAY CHAR =3D >>>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>>> AR=3D3D
>=3D3B=3D >>>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>>> buf: ARRAY [0..0]=3D >>>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>>> t=3D3B=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>>> Bnbsp=3D3D3B IF fromCha=3D >>>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>>> =3D3Bnbsp=3D3D
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B fromCharCac=3D >>>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>>> 3B<=3D3Bbr>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>>> arCache [c]<=3D3B=3D >>>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>>> romChar=3D3D3B<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>>> t=3D3B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>>> =3D3Br (c: CHAR): T =3D >>>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>>> >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>>> =3D3D3D NIL THEN&l=3D >>>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>>> =3D3D3D c=3D3D3B<=3D >>>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>>> 3D3D Text8.New (buf=3D >>>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>>> >=3D3B>=3D3B] :=3D >>>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>>> 3B>=3D3B&=3D3Bnb=3D >>>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>>> B >=3D3B>=3D3B<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>>> ishes before =3D >>>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>>> s made.<=3D3B=3D >>>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>>> catch these?=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>>> small epidemic.&l=3D >>>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>>> t=3D3B>=3D3Bize=3D >>>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>>> lt=3D3Bbr&=3D >>>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>>> the right construct=3D >>>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>>> B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>>> Bbr>=3D3B =3D >>>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>>> <=3D3B/html>=3D3B=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>>> 896e-68ee-44da-82dd=3D >>>>>>> -311cd8c58979_--
>>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/html; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>>
It is NOT benign.
The global = >>>> can be written before the initialization of what it points to is done.
T= >>>> he result is wrong.


It IS benign if multiple threads go and allo= >>>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>>
 =3B- Jay


&g= >>>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>>> r than benign? It's a cache and there's no problem if different threads get= >>>> different texts. Don't cripple performance unnecessarily. The collector w= >>>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>>> ly without actually getting written back to main memory.
>=3B >=3B P= >>>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>>> long time people didn't realize where all the compiler and proces=3D
&g= >>>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>>> v> >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- >> From dabenavidesd at yahoo.es Wed Aug 29 18:08:28 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 29 Aug 2012 17:08:28 +0100 (BST) Subject: [M3devel] Copyright clearance in Modula-3 books Message-ID: <1346256508.9030.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: how about asking for any part involved copyright clearance (compliance) with respect of SPwM3, M3 and AiM3. I knew of recent efforts here: http://thid.thesa.com/thid-0698-8201-th-1129-8613 Shouldn't we ask Greg Nelson for that? In that case we wouldn't need direct authorization to publish the Book, but just that, else stay tuned for a licensing of the material for us. Anyway if we have the sources of the book and nobody's claim that, couldn't be already cleared? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 2 09:28:40 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Message-ID: Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 2 09:38:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:38:52 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: References: Message-ID: okok..4.5 generates: ldx [%fp+2039], %g1 and %g1, 7, %g1 and %g1, 0xff, %g1 cmp %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: andcc %i0, 7, %g0 bne,pn %xcc, .LL2 - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 2 17:36:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 2 Aug 2012 16:36:40 +0100 (BST) Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: Message-ID: <1343921800.38422.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: but what pointers are you trying to diagnose, if it's a function closure then a REF to a RECORD paired with PROCEDURE is, that is, an OBJECT, but function "tagged" types, I don't think are M3language elements, that doesn't make them BRANDED by default, but structural equivalent by default just different for each type of procedure or method, if you think so, many languages have them, included C I guess, but why do we need those to build the compiler itself, perhaps object oriented environments can look for those in question, you can look for a hack B Kallsow tried to do in m3browser with RTType but I guess for structural types not branded types, so perhaps we need them both, it could be a nice work to do in static time, DEC experimented with some HW as a target for Cardelli's typeful Quest language, we could ask more about that experiment, since it targeted Modules static info and we could add everything you want in that. I need more time to do so, but looks interesting to do. Thanks in advance --- El jue, 2/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] unaligned procedures closure marker check broken? Para: "m3devel" Fecha: jueves, 2 de agosto, 2012 02:38 okok..4.5 generates: ??? ldx??? [%fp+2039], %g1 ??? and??? %g1, 7, %g1 ??? and??? %g1, 0xff, %g1 ??? cmp??? %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: ??? andcc??? %i0, 7, %g0 ??? bne,pn??? %xcc, .LL2 ?- Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89)???????? start_call_indirect type:addr (90)?????????? load_integer type:int_64 0 (91)?????????? pop_param type:int_64 (92)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93)?????????? loophole type1:addr type2:int_64 (94)?????????? load_integer type:int_64 7 (95)?????????? and type:int_64 (96)?????????? if_true type:int_64 label:1 frequency:0X32(50) (97)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98)?????????? load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: ?? ?save?? ?%sp, -208, %sp .LLCFI0: ?? ?stx?? ?%i0, [%fp+2175] ?? ?ldx?? ?[%fp+2175], %g1 ?? ?stx?? ?%g1, [%fp+2039] ?? ?ldx?? ?[%fp+2039], %g1 ?? ?and?? ?%g1, 7, %g1 <= correct ?? ?and?? ?%g1, 1, %g1? <= incorrect ?? ?and?? ?%g1, 0xff, %g1? <=? pointless, inefficient ?? ?cmp?? ?%g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { ? tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); ? EXPR_POP (); ? add_stmt (build3 (COND_EXPR, t_void, cond, ??????????????????? build1 (GOTO_EXPR, t_void, label), ??????????????????? NULL_TREE)); } static tree m3_cast (tree type, tree op0) { ? return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val;? true, false: Label;? freq: Frequency) = ? VAR skip := Next_label ();? nope := skip; ? BEGIN ??? IF (false # No_label) THEN? nope := false; END; ??? IF NOT Target.Aligned_procedures THEN ????? Push (proc); ????? Force (); ????? cg.loophole (Type.Addr, Target.Integer.cg_type); ????? Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); ????? cg.and (Target.Integer.cg_type); ???? ? ????? == insert comparisong to 0 here == ????? cg.if_true (Target.Integer.cg_type, nope, Always - freq); ????? SPop (1, "If_closure-unaligned"); ??? END; ?? ? and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 6 05:48:22 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 03:48:22 +0000 Subject: [M3devel] addresses of constants? Message-ID: Ugzip.m3 does an unnecessary heap allocation and copy. Here I have changed it to avoid the heap allocation. But it still makes a copy. How can I fix that? It seems ADR cannot be applied to constants? CONST ZLIB_VERSION = ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; This works but still is inefficient: PROCEDURE deflateInit(strm: z_stream_star; level: int): int = VAR version := ZLIB_VERSION; BEGIN RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); END deflateInit; I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 6 19:40:21 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 06 Aug 2012 12:40:21 -0500 Subject: [M3devel] addresses of constants? In-Reply-To: References: Message-ID: <50200185.1060106@lcwb.coop> You probably already know this, but, from 2.7: ADR (VAR x: Any) : ADDRESS ... ADR(x) is the address of the variable x. The actual argument must be a designator but need not be writable. And from 2.6.3: An identifier is a writable designator if it is declared as a variable, is a VAR or VALUE parameter, is a local of a TYPECASE or TRY-EXCEPT statement, or is a WITH local that is bound to a writable designator. An identifier is a readonly designator if it is a READONLY parameter, a local of a FOR statement, or a WITH local bound to a nondesignator or readonly designator. I'm speculating, but confidently, that the reason for the designator restriction is to have an implementation-independent rule that ensures the value would be in memory, and thus have an address, regardless of whether ADR is applied to it or not, The ADR could happen in a separate compilation from the declaration, and its existence be unknown at least until the main program was compiled. This way, compiler writers have the option of putting any constant in immediate instruction fields, if the target machine has them, without risk of undermining source code that compiled and ran fine on a different machine or with a different compiler. Even if it's sure to be in memory, allowing ADR(SomeConstant) also would open up the possibility of a constant that wasn't constant, a la FORTRAN. Yeah, I know it's already unsafe code anyway, but that would be an additional form of unsafety with scarcely an imaginable benefit. I'd just make it a VAR and put it inside a module. You would only have to verify that that one module didn't change it. It's possible a smart enough compiler would initialize right in the variable in the executable and not have to copy it. BTW, VAR ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) appears to have another problem. It assumes the representation of "1.0.4" is the same as, or at least contains ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; which it also looks like your modified UgzipP.deflateInit would expect. This happens to be true in pm3, but not cm3. Without debating the merits of various representations of TEXT literals, it would make for implementation-dependent code in any case. On 08/05/2012 10:48 PM, Jay K wrote: > Ugzip.m3 does an unnecessary heap allocation and copy. > > > Here I have changed it to avoid the heap allocation. > But it still makes a copy. > > How can I fix that? > > It seems ADR cannot be applied to constants? > > CONST > ZLIB_VERSION = ARRAY [0..5] OF char{ > ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; > > This works but still is inefficient: > > PROCEDURE deflateInit(strm: z_stream_star; level: int): int = > VAR version := ZLIB_VERSION; > BEGIN > RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); > END deflateInit; > > > I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); > const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. > > > - Jay From jay.krell at cornell.edu Tue Aug 7 01:02:16 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 23:02:16 +0000 Subject: [M3devel] initializer problem? In-Reply-To: References: <20120806105804.20F9F2474003@birch.elegosoft.com>, , , Message-ID: I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: DepthArray = REF ARRAY OF RECORD n := 0 END; dcount := NEW(DepthArray, 10); FOR i := FIRST(dcount^) TO LAST(dcount^) DO <* ASSERT dcount[i].n = 0 *> END; the assertion fails sometimes. I'll have to keep digging. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3 17 Mar 2008 03:46:14 -0000 1.2 +++ VBTRep.m3 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ (* All d: dcount[d].n = # VBTs in list with depth at most d. *) WITH v = NEW(REF ARRAY OF VBT.T, n) DO FOR i := 0 TO n - 1 DO + IF list[i].depth > 0 THEN v[dcount[list[i].depth - 1].n] := list[i].v; INC(dcount[list[i].depth - 1].n); (* All d: dcount[d-1].n = # VBTs in list with depth < d, or with depth = d that have been copied into v. *) END; + END; (* v has all the VBTs in the list and is sorted by depth *) FOR i := 0 TO n - 1 DO IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Tue Aug 7 15:44:27 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:44:27 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211BBB.4070203@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From mfleischer at elego.de Tue Aug 7 15:51:25 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:51:25 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211D5D.2040901@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Tue Aug 7 16:28:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 15:28:27 +0100 (BST) Subject: [M3devel] [cm3 trac maintenance notification] In-Reply-To: <50211BBB.4070203@elego.de> Message-ID: <1344349707.21021.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi: how much time, will be down, or just instantly be migrated? Well my hope is all bugs were gone but ... Thanks in advance --- El mar, 7/8/12, Maurizio Fleischer escribi?: De: Maurizio Fleischer Asunto: [M3devel] [cm3 trac maintenance notification] Para: m3devel at elego.de, m3announce at elego.de CC: admin at elego.de Fecha: martes, 7 de agosto, 2012 08:44 Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services&? Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96? ? ? maurizio.fleischer at elegosoft.com fax???+49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 7 19:50:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 18:50:31 +0100 (BST) Subject: [M3devel] initializer problem? In-Reply-To: Message-ID: <1344361831.31130.YahooMailClassic@web29701.mail.ird.yahoo.com> Hi all: interestingly, Greg Nelson mentioned here: http://books.google.com.co/books?id=EBW2AAAAIAAJ&q=trestle+%22dec-SRC%22&dq=trestle+%22dec-SRC%22&source=bl&ots=RTsXmdey5q&sig=XENEwmkvSMW3BzyeN9w_68VdcHU&hl=en&redir_esc=y that Trestle had some problem in RISC architectures, I wonder what he was referring to, and if they were working in a new design what did they changed. Thanks in advance --- El lun, 6/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] initializer problem? Para: "m3devel" Fecha: lunes, 6 de agosto, 2012 18:02 I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: ? DepthArray = REF ARRAY OF RECORD n := 0 END; ????? dcount := NEW(DepthArray, 10); ????? FOR i := FIRST(dcount^) TO LAST(dcount^) DO ??????? <* ASSERT dcount[i].n = 0 *> ????? END; the assertion fails sometimes. I'll have to keep digging. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3??? 17 Mar 2008 03:46:14 -0000??? 1.2 +++ VBTRep.m3??? 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ ?????? (* All d: dcount[d].n = # VBTs in list with depth at most d. *) ?????? WITH v = NEW(REF ARRAY OF VBT.T, n) DO ???????? FOR i := 0 TO n - 1 DO +????????? IF list[i].depth > 0 THEN ?????????? v[dcount[list[i].depth - 1].n] := list[i].v; ?????????? INC(dcount[list[i].depth - 1].n); ?????????? (* All d: dcount[d-1].n = # VBTs in list with depth < d, ????????????? or with depth = d that have been copied into v. *) ???????? END; +??????? END; ???????? (* v has all the VBTs in the list and is sorted by depth *) ???????? FOR i := 0 TO n - 1 DO ?????????? IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Wed Aug 8 16:00:14 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Wed, 08 Aug 2012 10:00:14 -0400 Subject: [M3devel] [cm3 trac maintenance notification] project back online In-Reply-To: <50211D5D.2040901@elego.de> References: <50211D5D.2040901@elego.de> Message-ID: <502270EE.80404@elego.de> Hi, the cm3 trac project is online again. As mentioned it is now reachable at https://cm3-bugs.elegosoft.com/cm3/. links in old ticket mails are redirected. kind regards elego systems administration On 08/07/2012 09:51 AM, Maurizio Fleischer wrote: > Hello, > due to maintenance work the cm3 trac project will be down tomorrow > 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is > reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users > accessing > https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make > your important changes beforehand and log out. > We apologize for the inconvenience. > > Thanks > elego systems administration > -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Sat Aug 11 16:20:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:20:40 +0100 (BST) Subject: [M3devel] Elegosoft.com mailer down? Message-ID: <1344694840.84378.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: How may times in a row could I send an malformed email? May I miss something m3devel ? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Aug 11 16:14:41 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:14:41 +0100 (BST) Subject: [M3devel] About extended char types, and integer types Message-ID: <1344694481.85507.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: I'm getting serious about this so I hope you don't misunderstand me, see this two differences in point of view of Modula-3 char character type and integer type systems: http://rant.gulbrandsen.priv.no/code/modula3-integers http://slps.svn.sourceforge.net/viewvc/slps/topics/grammars/modula/src-052/src.syntax.txt?view=markup&pathrev=1078 Though in some form, Modula-3 is better than this kind of things today are written,? if somebody takes the time to talk about it, it makes me think that something went wrong with today's software, perhaps that software is already dead. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 15 05:04:50 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 03:04:50 +0000 Subject: [M3devel] reference to globals in globals? Message-ID: Isn't it safe to take the address of a global? I have something like this: CONST UID_INTEGER = 1234; CONST UID_FLOAT = 4567; ... several more ... TYPE CType = OBJECT .. END; VAR t_int: CType := ...; VAR t_float: CType := ...; ... several more ... MapTypeIdToType(UID_INTEGER, t_int); MapTypeIdToType(UID_FLOAT, FLOAT); ... several more ... but what I really want is more like: TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: REF CType; END; CONST BuiltinUids = ARRAY OF BuiltinUids { BuiltinUids{UID_INTEGER, &t_int}, BuiltinUids{UID_FLOAT, &t_float}, ... several more ... }; FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); END; Heck, even if these weren't global, is it that unreasonble, from the programmer's point of view, for the language/compiler to do some pointer escape analysis and let me take the address of a local, as long as I don't store it somewhere that outlives the local? You can see this particular pattern currently in m3-sys/m3cc/gcc/gcc/m3cg/parse.c and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 where I encounter this. Working in safe languages can be frustrating... Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 15 16:51:41 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:51:41 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: Message-ID: <502BB77D.1030400@lcwb.coop> On 08/14/2012 10:04 PM, Jay K wrote: > Isn't it safe to take the address of a global? > Do you mean can't you use the ADR function in safe code if you apply it only to a global variable? The answer to that is no. The ADR function is illegal altogether in safe code. As to why, I can only speculate, but see below. I suspect even in this case, it is not as simple as it seems. > > I have something like this: > > > CONST UID_INTEGER = 1234; > CONST UID_FLOAT = 4567; > ... several more ... > > > TYPE CType = OBJECT .. END; > > > VAR t_int: CType := ...; > VAR t_float: CType := ...; > ... several more ... > > > MapTypeIdToType(UID_INTEGER, t_int); > MapTypeIdToType(UID_FLOAT, FLOAT); > ... several more ... > > > but what I really want is more like: > > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: REF CType; ^UNTRACED REF? If it were just REF, that would imply that your global variable (the pointer it contains) is a heap object, that it has heap allocator/GC overhead data attached to it, and that the GC should trace it, none of which is true. > END; > > > CONST BuiltinUids = ARRAY OF BuiltinUids { > BuiltinUids{UID_INTEGER, &t_int}, > BuiltinUids{UID_FLOAT, &t_float}, ADR instead of &? If so, you are still not there, because ADR returns a value of type ADDRESS, i.e., an untraced reference to we-don't-know-what. Somewhere, you would also have to use a LOOPHOLE to get it to UNTRACED REF CType. > ... several more ... > }; > > > FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > END; > I don't know what the signature of MapTypeIdToType is, but above, you are passing a variable of object type to its 2nd parameter, (which contains a traced reference to the actual heap object). But here, you pass the _address_ of the above. Inconsistent number of levels of indirection. A static safe language is much more likely to help with things like this. Maybe you just want to say TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: CType; and BuiltinUids{UID_INTEGER, t_int}? This would be equivalent to your first way, and doesn't require any unsafe coding at all. Or, you could do away with global variable t_int altogether and just initialize directly into BuiltinUids[..].ctype with whatever expression you used to initialize t_int. It looks like your array makes the t_int and cousins redundant. > > Heck, even if these weren't global, is it that unreasonble, > from the programmer's point of view, for the language/compiler > to do some pointer escape analysis and let me take the address > of a local, as long as I don't store it somewhere that outlives > the local? > This is ultimately an undecidable problem and even conservative approximations of reasonable sophistication are far too involved for a language to require of every compiler. > > You can see this particular pattern currently in > m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > > and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > where I encounter this. > > > Working in safe languages can be frustrating... > It's just an instant/deferred gratification thing. Safe languages often make you stop and fix it before you run it. Unsafe languages let you naively forge ahead to the next step, where the bug is likely to be *much* harder to diagnose, assuming you even have enough test cases to notice it at all during development. Your code here is a good example. Of course, safe languages occasionally make you unnecessarily write a bit more code to do it the safe way. E.g., the famous fake pointer to the root of a linked list example. In my personal experience, these times are at least one order of magnitude less frequent than the times safe languages reduce great pain to minor pain, albeit sooner. If fact, if you are accustomed to thinking in type-safe terms, it is seldom any harder to code it safely in the first place. You're making it too difficult. > > Thank you, > - Jay From rodney_bates at lcwb.coop Wed Aug 15 16:57:49 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:57:49 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20120815043215.C8614CC85C@birch.elegosoft.com> References: <20120815043215.C8614CC85C@birch.elegosoft.com> Message-ID: <502BB8ED.7030108@lcwb.coop> On 08/15/2012 06:32 AM, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/15 06:32:15 > > Modified files: > ./: Make-lang.in > > Log message: > compile parse.c as C++ even if compiling the rest of gcc as C (or however > it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in > (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes > too much to compile as C++, plus I'm not likely to fight this battle with 4.7) Does this mean we will now have to have a C++ compiler for every target we want M3 on? > > Modified files: > cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 > > Log message: > flush some test code > > From hosking at cs.purdue.edu Wed Aug 15 18:11:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 12:11:57 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: Jay, Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. I don?t know enough about your use-case to understand what you are trying to do. On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: >> Isn't it safe to take the address of a global? >> > > Do you mean can't you use the ADR function in safe code > if you apply it only to a global variable? The answer > to that is no. The ADR function is illegal altogether in > safe code. > > As to why, I can only speculate, but see below. I suspect > even in this case, it is not as simple as it seems. > >> >> I have something like this: >> >> >> CONST UID_INTEGER = 1234; >> CONST UID_FLOAT = 4567; >> ... several more ... >> >> >> TYPE CType = OBJECT .. END; >> >> >> VAR t_int: CType := ...; >> VAR t_float: CType := ...; >> ... several more ... >> >> >> MapTypeIdToType(UID_INTEGER, t_int); >> MapTypeIdToType(UID_FLOAT, FLOAT); >> ... several more ... >> >> >> but what I really want is more like: >> >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: REF CType; > > ^UNTRACED REF? If it were just REF, that would imply that > your global variable (the pointer it contains) is a heap object, that > it has heap allocator/GC overhead data attached to it, and that the GC > should trace it, none of which is true. > > >> END; >> >> >> CONST BuiltinUids = ARRAY OF BuiltinUids { >> BuiltinUids{UID_INTEGER, &t_int}, >> BuiltinUids{UID_FLOAT, &t_float}, > > ADR instead of &? If so, you are still not there, because ADR > returns a value of type ADDRESS, i.e., an untraced reference to > we-don't-know-what. Somewhere, you would also have to use a > LOOPHOLE to get it to UNTRACED REF CType. > >> ... several more ... >> }; >> >> >> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >> END; >> > > I don't know what the signature of MapTypeIdToType is, but above, > you are passing a variable of object type to its 2nd parameter, > (which contains a traced reference to the actual heap object). > But here, you pass the _address_ of the above. Inconsistent > number of levels of indirection. A static safe language is > much more likely to help with things like this. > > Maybe you just want to say > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: CType; > > and > > BuiltinUids{UID_INTEGER, t_int}? > > This would be equivalent to your first way, and doesn't require any > unsafe coding at all. > > Or, you could do away with global variable t_int altogether and > just initialize directly into BuiltinUids[..].ctype with whatever > expression you used to initialize t_int. It looks like your array > makes the t_int and cousins redundant. > >> >> Heck, even if these weren't global, is it that unreasonble, >> from the programmer's point of view, for the language/compiler >> to do some pointer escape analysis and let me take the address >> of a local, as long as I don't store it somewhere that outlives >> the local? >> > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. > >> >> You can see this particular pattern currently in >> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >> >> >> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >> where I encounter this. >> >> >> Working in safe languages can be frustrating... >> > > It's just an instant/deferred gratification thing. Safe languages often > make you stop and fix it before you run it. Unsafe languages let you naively > forge ahead to the next step, where the bug is likely to be *much* harder to > diagnose, assuming you even have enough test cases to notice it at all during > development. Your code here is a good example. > > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. In my personal experience, these times are at least > one order of magnitude less frequent than the times safe languages reduce > great pain to minor pain, albeit sooner. If fact, if you are accustomed to > thinking in type-safe terms, it is seldom any harder to code it safely in the > first place. > > You're making it too difficult. > >> >> Thank you, >> - Jay > From jay.krell at cornell.edu Wed Aug 15 18:56:09 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:56:09 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: <502BB77D.1030400@lcwb.coop> Message-ID: I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable? The answer >> to that is no. The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below. I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>> ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>> ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>> ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>> typeid: INTEGER; >>> ctype: REF CType; >> >> ^UNTRACED REF? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>> BuiltinUids{UID_INTEGER, &t_int}, >>> BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what. Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>> ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above. Inconsistent >> number of levels of indirection. A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int. It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing. Safe languages often >> make you stop and fix it before you run it. Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development. Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > From jay.krell at cornell.edu Wed Aug 15 18:50:04 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:50:04 -0400 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <502BB8ED.7030108@lcwb.coop> References: <20120815043215.C8614CC85C@birch.elegosoft.com> <502BB8ED.7030108@lcwb.coop> Message-ID: <34072368-7504-443C-B5EE-C0E75A75A68D@gmail.com> Yes that has already been the case for a long time. Gcc itself is also moving that way but isn't there yet. C++ compilers have long been widespread. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 10:57 AM, "Rodney M. Bates" wrote: > > > On 08/15/2012 06:32 AM, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 12/08/15 06:32:15 >> >> Modified files: >> ./: Make-lang.in >> >> Log message: >> compile parse.c as C++ even if compiling the rest of gcc as C (or however >> it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in >> (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes >> too much to compile as C++, plus I'm not likely to fight this battle with 4.7) > > Does this mean we will now have to have a C++ compiler for every target we want > M3 on? > >> >> Modified files: >> cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 >> >> Log message: >> flush some test code >> >> > From dabenavidesd at yahoo.es Wed Aug 15 21:46:39 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 15 Aug 2012 20:46:39 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345059808.37105.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <1345059999.77923.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: we need to calculate first Module instantiation in Modula-3 in SPIN MODULE_UNIT, that is trying to establish first Module instantiation, initialization, and then you can extract CONST "address" in the actual case is similar to that, a pointer to an address of the actual value, if pointer is NIL valued then it needed initialization in compile time else is a RT error to use its value, as we have proposed in other cases. Jay, please see this BitC language, has constructors for all those RT structures of your new back-end (hope you enjoy it, see p. 14, s. 3.6.2, Tag Representation): http://bitc-lang.org/docs/bitc/spec.pdf ). This leads in some special architectures to a double pointer-sized machine word type, which is why I talked about LONGADDRESS proposal. Thanks in advance --- El mi?, 15/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] reference to globals in globals? Para: "Tony Hosking" CC: "m3devel at elegosoft.com developers" , "Jay K" Fecha: mi?rcoles, 15 de agosto, 2012 11:56 I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array.? More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable?? The answer >> to that is no.? The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below.? I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>>? ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>>? ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>>? ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>>? typeid: INTEGER; >>>? ctype: REF CType; >> >>? ? ? ? ???^UNTRACED REF?? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>>? BuiltinUids{UID_INTEGER, &t_int}, >>>? BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &?? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what.? Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>>? ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>>? MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above.? Inconsistent >> number of levels of indirection.? A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >>? typeid: INTEGER; >>? ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int.? It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing.? Safe languages often >> make you stop and fix it before you run it.? Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development.? Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way.? E.g., the famous fake pointer to the root of a >> linked list example.? In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner.? If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:07:20 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:07:20 +0000 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: > If fact, if you are accustomed to thinking in type-safe terms I do think in fairly type-terms. In C++ I know to use templates, for example. There is a matter of lifetime-safety though. I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. Consider C++ code like this, where I want to do some things "the same" to a bunch of variables, and some things "specific". struct Person_t { const char* name; }; void F() { Person_t jay {"jay"}; Person_t randy = {"randy"}; Person_t tony = {"tony"}; // print all their names Person_t* people[] = {&jay, &randy, &tony}; for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i]->name); DoSomethingSpecificToJay(&jay); DoSomethingSpecificToRandy(&randy); DoSomethingSpecificToTony(&tony); } I do NOT want to do this: void F() { Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; // print all their names for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i].name); DoSomethingSpecificToJay(&people[0]); DoSomethingSpecificToRandy(&people[1]); DoSomethingSpecificToTony(&people[2]); } I understand there is: enum People { jay, randy, tony }; ... DoSomethingSpecificToJay(&people[jay]); DoSomethingSpecificToRandy(&people[randy]); DoSomethingSpecificToTony(&people[tony]); ... but I'm still not thrilled about that. I ended up without anything "specific" except initialization..so no really need to hang a name/variable on the data, and the initialization is fairly specific, so the resulting code is merely: EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); ... you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems in the m3cg interface...these types need to be defined via function calls into the backend, not hardcoded..and even if they are...the order of types arriving at the backend isn't always ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) - Jay > From: jay.krell at cornell.edu > Date: Wed, 15 Aug 2012 12:56:09 -0400 > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] reference to globals in globals? > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > Jay, > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/14/2012 10:04 PM, Jay K wrote: > >>> Isn't it safe to take the address of a global? > >>> > >> > >> Do you mean can't you use the ADR function in safe code > >> if you apply it only to a global variable? The answer > >> to that is no. The ADR function is illegal altogether in > >> safe code. > >> > >> As to why, I can only speculate, but see below. I suspect > >> even in this case, it is not as simple as it seems. > >> > >>> > >>> I have something like this: > >>> > >>> > >>> CONST UID_INTEGER = 1234; > >>> CONST UID_FLOAT = 4567; > >>> ... several more ... > >>> > >>> > >>> TYPE CType = OBJECT .. END; > >>> > >>> > >>> VAR t_int: CType := ...; > >>> VAR t_float: CType := ...; > >>> ... several more ... > >>> > >>> > >>> MapTypeIdToType(UID_INTEGER, t_int); > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > >>> ... several more ... > >>> > >>> > >>> but what I really want is more like: > >>> > >>> > >>> TYPE RECORD = BuiltinUid_t = > >>> typeid: INTEGER; > >>> ctype: REF CType; > >> > >> ^UNTRACED REF? If it were just REF, that would imply that > >> your global variable (the pointer it contains) is a heap object, that > >> it has heap allocator/GC overhead data attached to it, and that the GC > >> should trace it, none of which is true. > >> > >> > >>> END; > >>> > >>> > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > >>> BuiltinUids{UID_INTEGER, &t_int}, > >>> BuiltinUids{UID_FLOAT, &t_float}, > >> > >> ADR instead of &? If so, you are still not there, because ADR > >> returns a value of type ADDRESS, i.e., an untraced reference to > >> we-don't-know-what. Somewhere, you would also have to use a > >> LOOPHOLE to get it to UNTRACED REF CType. > >> > >>> ... several more ... > >>> }; > >>> > >>> > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > >>> END; > >>> > >> > >> I don't know what the signature of MapTypeIdToType is, but above, > >> you are passing a variable of object type to its 2nd parameter, > >> (which contains a traced reference to the actual heap object). > >> But here, you pass the _address_ of the above. Inconsistent > >> number of levels of indirection. A static safe language is > >> much more likely to help with things like this. > >> > >> Maybe you just want to say > >> > >> TYPE RECORD = BuiltinUid_t = > >> typeid: INTEGER; > >> ctype: CType; > >> > >> and > >> > >> BuiltinUids{UID_INTEGER, t_int}? > >> > >> This would be equivalent to your first way, and doesn't require any > >> unsafe coding at all. > >> > >> Or, you could do away with global variable t_int altogether and > >> just initialize directly into BuiltinUids[..].ctype with whatever > >> expression you used to initialize t_int. It looks like your array > >> makes the t_int and cousins redundant. > >> > >>> > >>> Heck, even if these weren't global, is it that unreasonble, > >>> from the programmer's point of view, for the language/compiler > >>> to do some pointer escape analysis and let me take the address > >>> of a local, as long as I don't store it somewhere that outlives > >>> the local? > >>> > >> > >> This is ultimately an undecidable problem and even conservative > >> approximations of reasonable sophistication are far too involved > >> for a language to require of every compiler. > >> > >>> > >>> You can see this particular pattern currently in > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > >>> > >>> > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > >>> where I encounter this. > >>> > >>> > >>> Working in safe languages can be frustrating... > >>> > >> > >> It's just an instant/deferred gratification thing. Safe languages often > >> make you stop and fix it before you run it. Unsafe languages let you naively > >> forge ahead to the next step, where the bug is likely to be *much* harder to > >> diagnose, assuming you even have enough test cases to notice it at all during > >> development. Your code here is a good example. > >> > >> Of course, safe languages occasionally make you unnecessarily write a bit more > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > >> linked list example. In my personal experience, these times are at least > >> one order of magnitude less frequent than the times safe languages reduce > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > >> first place. > >> > >> You're making it too difficult. > >> > >>> > >>> Thank you, > >>> - Jay > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:11:25 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:11:25 +0000 Subject: [M3devel] m3 backends and builtin types? Message-ID: Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:15:51 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:15:51 +0000 Subject: [M3devel] some proposed m3cg additions? Message-ID: I suggest backends should be informed of the following somehow via m3cg calls: word size definitely endian maybe jmpbuf size maybe target_name maybe I understand they are available from m3middle. Part of my agenda is that .mc files be more self contained. Eventually I'd like the IL to be independent of all of these anyway, but that is a later project. Jmpbuf size I know how to eliminate. Target_name doesn't really have meaning. Eliminating word size and endian dependencies in the IL won't be so easy. The extra information should be pretty harmless even if not critical. For example though...if you look at m3cgcat..it is slightly bad that it has to discover target from the ambient environment, instead of from the input file. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 01:58:23 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 19:58:23 -0400 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: Message-ID: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote: > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so, RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 16 02:55:12 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 15 Aug 2012 20:55:12 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: <20120816005512.GA31309@topoi.pooq.com> On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures? the ones whose environments are completely static? Or am I thinking of another type-safe language? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. You mean something like (in C with probable errors) list **p, *q; make q point to some linked list for ( p = &q; *p != NIL; p = &((*p)->next)) { And here you can easily delete elements by *p = (*p) -> next } This can be done prefectly type-safe in languages that allow REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik From dabenavidesd at yahoo.es Thu Aug 16 03:14:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 02:14:27 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> Message-ID: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html Thanks in advance --- El mi?, 15/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] reference to globals in globals? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 15 de agosto, 2012 19:55 On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures?? the ones whose environments are completely static? Or am I thinking of another type-safe language?? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way.? E.g., the famous fake pointer to the root of a > linked list example.? You mean something like (in C with probable errors) ? list **p, *q; ? make q point to some linked list ? for ( p = &q; *p != NIL; p = &((*p)->next)) ? { ???And here you can easily delete elements by *p = (*p) -> next ? } This can be done prefectly type-safe in languages that allow ???REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 04:07:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 02:07:52 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: > There is a bootstrap issue here isn?t there? I am not sure. I don't understand the system enough. :( I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. The compiler does for example construct the type BOOLEAN out of lower level pieces. Can it not then proceed to compute its typeid, and inform the backend about it? Other examples appear to include...wait.where did I get this list? At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). I built up the following list: UID_INTEGER = 16_195C2A74; (* INTEGER *) UID_LONGINT = 16_05562176; (* LONGINT *) UID_WORD = 16_97E237E2; (* CARDINAL *) UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) UID_REEL = 16_48E16572; (* REAL *) UID_LREEL = 16_94FE32F6; (* LONGREAL *) UID_XREEL = 16_9EE024E3; (* EXTENDED *) UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) UID_CHAR = 16_56E16863; (* CHAR [0..255] *) UID_WIDECHAR = 16_88F439FC; UID_MUTEX = 16_1541F475; (* MUTEX *) UID_TEXT = 16_50F86574; (* TEXT *) UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) UID_ROOT = 16_9D8FB489; (* ROOT *) UID_REFANY = 16_1C1C45E6; (* REFANY *) UID_ADDR = 16_08402063; (* ADDRESS *) UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) UID_NULL = 16_48EC756E; (* NULL *) Surely these aren't all fundamental?? Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: TYPE Record1 = RECORD REF Record2 END; TYPE Record2 = RECORD REF Record1 END; If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. - Jay From: hosking at cs.purdue.edu Date: Wed, 15 Aug 2012 19:58:23 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3 backends and builtin types? There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote:Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 06:23:37 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 04:23:37 +0000 Subject: [M3devel] m3middle/TFloat Message-ID: This TFloat module doesn't look right to me. I guess, ok, in a sense -- if host floating point is adequate.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 15:58:59 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 13:58:59 +0000 Subject: [M3devel] m3cg.set_target_name, set_endian, set_wordsize? Message-ID: I already said this, but more concretely, I propose M3CG_Ops additions: set_target_name(name: TEXT); TYPE Endian = { Little, Big }; (* or Endian_t or Endian_e *) set_endian(endian: Endian); set_wordsize(wordsize: INTEGER); (* currently always 32 or 64 *) I MIGHT have use of these in m3back/M3C. I'm not sure yet. target_name implies endian and wordsize and more, but it a kind of vague thing. It also implies win32 vs. Posix, which M3C probably won't need. It also implies jmpbuf size, which might be useful, or maybe we remove that "the other way". You know -- I want to eventually remove target knowledge from m3front, and have none in M3C either. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 16:21:01 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 14:21:01 +0000 Subject: [M3devel] higher level m3cg? Message-ID: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 17:06:18 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Thu, 16 Aug 2012 11:06:18 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Yes, I've thought so for a long time. It would also more easily enable llvm. Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 17:53:58 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 16:53:58 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345132438.30720.YahooMailClassic@web29701.mail.ird.yahoo.com> HI all: OO C is C++ in theory not in practice, could be better than this, but we should not do so and don't look a Modula-3 type system that preserves under compilation that's an abstract machine that interacts with whatever machine code generator is (M3CG) compiled to for whatever language you would like? That's why I insist the original acronym of M3CG(M2CG) is Machine Code Generator, this is not a translator, nor anything else, it was created for the purpose of compiling and linking compiler intermediate language, nothing else. Besides many languages and tongues I guess is better to speak clearly in a universal machine like Baby Modula-3, choosing Modula-3 as mother tongue, that is we need to speak in Modula-3 from Baby and counter wise. As some TV add said: "the problem is simple the answer it's not" So I would try to explain meaningfully clearly, we need to understand how to encode Modula-3 (1989) in a language like Quest (1994) in terms of Baby Modula-3 (1993), that is Target is BM3, Host is Quest, and Source is Modula-3. It turns out that doing type inference in terms of Quest is decidable, but in terms of Baby Modula-3 I don't know, it's pretty much a question of the million, the problem for real is can we encode Modula-3 in Baby Modula-3? That's a good question for the people who advocate for not so obscure language definition (It has some examples of understanding the language in terms of a exception key concept, but not in object terms) So, what is next is to find a theory of Objects, there are people that is not happy with Cardelli's definition and I agree with them we should we looking at the problem concretely with options I know of different kinds of options, but you would not like to hear the one I prefer is likely to be undecidable and inconsistent terms, so we could argue about obscurity of language definition again (right ?). This option is called pebble is a language written by Butler Lampson (Modula-2+ author) and Rodney Burstall (a aforementioned computer theorist ). In some sense Pebble is just about to talk about Modula-3 Module system, but it would need a lot of research to put it on place for being chosen. Thanks in advance --- El jue, 16/8/12, Jay K escribi?: De: Jay K Asunto: [M3devel] higher level m3cg? Para: "m3devel" Fecha: jueves, 16 de agosto, 2012 09:21 Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 16 18:40:45 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 11:40:45 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> References: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <502D228D.7070306@lcwb.coop> On 08/15/2012 08:14 PM, Daniel Alejandro Benavides D. wrote: > Hi all: > For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" > http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html > > Thanks in advance > I think here, "top" means outermost in the nested scopes of a compilation. That is, a "top-level" procedure is declared immediately inside an interface or module, not inside any procedure or block. This is an entirely different hierarchy from that of interfaces importing other interfaces, where the "bottom of the world" phrase is used to mean these types do not refer to any other imported interface. Oddly, I have heard this second sense referred to as the "top" of a type hierarchy, but that usage is highly counterintuitive to me. > --- El *mi?, 15/8/12, Hendrik Boom //* escribi?: > > > De: Hendrik Boom > Asunto: Re: [M3devel] reference to globals in globals? > Para: m3devel at elegosoft.com > Fecha: mi?rcoles, 15 de agosto, 2012 19:55 > > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > > > >Heck, even if these weren't global, is it that unreasonble, > > >from the programmer's point of view, for the language/compiler > > >to do some pointer escape analysis and let me take the address > > >of a local, as long as I don't store it somewhere that outlives > > >the local? > > > > > > > This is ultimately an undecidable problem and even conservative > > approximations of reasonable sophistication are far too involved > > for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > > > Of course, safe languages occasionally make you unnecessarily write a bit more > > code to do it the safe way. E.g., the famous fake pointer to the root of a > > linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } > > This can be done prefectly type-safe in languages that allow > > REF REF list > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Thu Aug 16 19:27:17 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 12:27:17 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502D2D75.8010702@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > Yes, yes, yes!!! > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? Yes. If you can assign a procedure (or pointer to function, which is equivalent, for this discussion) to a variable, then there are ways to call the procedure when its environment does not exist, i.e, one or more scopes of variables that the procedure is declared inside of and can refer to. The outermost scope always exists, so we need three nested scopes to get an example. MODULE Outer ; PROCEDURE Middle ( MiddleFormal : INTEGER ) = PROCEDURE Inner ( ) : INTEGER = BEGIN RETURN MiddleFormal END ; BEGIN (* Middle *) ProcVar := Inner (* Statically illegal because Inner is not top-level. *) END (* Middle ; VAR ProcVar : PROCEDURE ( ) : INTEGER ; VAR X : INTEGER ; BEGIN (* Outer *) Middle ( 7 ) ; X := ProcVar ( ) END Outer . If the assignment ProcVar := Inner were legal, the call ProcVar ( ) goes to Inner, which refers to MiddleFormal of Middle, but there is no active invocation of Middle, and MiddleFormal doesn't exist. Pascal (if I remember all this right) allows procedures to be passed as parameters, but not assigned to variables. The procedure constant in question can only be named where it has an environment. If all you can do with it is pass it, any use of it can only be deeper inside the dynamic call chain, where its environment continues to exist. Modula-2 allows procedures to be either passed or assigned, but in either case, only a top-level procedure, which always has an environment, can be given. Both passing and assigning procedures have reasonable uses, both for variations on the callback theme. Modula-3 supports both by allowing any procedure to be passed but only a top-level one to be assigned. Enforcing the latter restriction sometimes means a runtime check at the point of an assignment, if the RHS of the assignment is a procedure formal parameter, which could dynamically be top-level or not. This is probably not a high-frequency situation. Assigning a procedure constant and passing any procedure can be fully checked at compile time. I think it is possible to define a weaker, yet safe rule that postpones runtime environment existence checking until the point of a call on a procedure variable, but I don't recall seeing this in any language. Maybe Algol68 or something. In gcc-enhanced C, which allows nested functions, in addition to both assigning and passing (pointers to) functions, you are on your own to avoid calling something that lacks an environment. The one help the language gives is to say in the reference, "all hell will break loose" if you get this wrong. > > -- hendrik From rodney_bates at lcwb.coop Thu Aug 16 20:16:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 13:16:35 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502D3903.6020606@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: This is from memory, is incomplete, and my not be 100% reliable, but hopefully useful. There are two different hash codes on type structures used in (at least) the compiler, runtime, and pickles. The "uid" is 32-bit, and the "signature" is 64-bit. I think the signature maybe only applies to reference types. It is what is used in pickles. One of them takes into account some properties the other does not. I am not entirely sure which of the following information applies to which hash code(s). At one time, I think I had found a main program that computed one or the other of these, or both. They are not arbitrary, they are somehow derived from the actual type structures. (What are the type structures for primitive builtin types?) For builtin types, they have apparently been hand copied to several places. (compiler, RTS, pickles, m3gdb, ...?) They also get computed for programmer-defined types too, but obviously you won't find these hard-coded. However, some of the values for builtin types have suffered some alterations. Here is an excerpt from m3gdb, m3-lang.c, with a possibly relevant comment that I once inferred somehow: if (m3uid_to_num (uid, &num_uid)) { if (num_uid == 0x195c2a74) { return builtin_type_m3_integer; } else if (num_uid == 0x05562176) { return builtin_type_m3_longint; } else if (num_uid == 0x50f86574) { return builtin_type_m3_text; } else if (num_uid == 0x97e237e2) { return builtin_type_m3_cardinal; } else if (num_uid == 0x1e59237d) { return builtin_type_m3_boolean; } else if (num_uid == 0x08402063) { return builtin_type_m3_address; } else if (num_uid == 0x9d8fb489) { return builtin_type_m3_root; } else if (num_uid == 0xa973d3a6) { return builtin_type_m3_transient_root; } else if (num_uid == 0x56e16863) { return builtin_type_m3_char; } /* For widechar, the num_uid was once 0xb0830411. Presumably, this is an outdated leftover from a transitional implementation of widechar, at Critical Mass, before the final one came from them. */ else if (num_uid == 0x88f439fc) { return builtin_type_m3_widechar; } else if (num_uid == 0x48e16572) { return builtin_type_m3_real; } else if (num_uid == 0x94fe32f6) { return builtin_type_m3_longreal; } else if (num_uid == 0x9ee024e3) { return builtin_type_m3_extended; } else if (num_uid == 0x48ec756e) { return builtin_type_m3_null; } else if (num_uid == 0x1c1c45e6) { return builtin_type_m3_refany; } else if (num_uid == 0x51e4b739) { return builtin_type_m3_transient_refany; } else if (num_uid == 0x898ea789) { return builtin_type_m3_untraced_root; } else if (num_uid == 0x00000000) { return builtin_type_m3_void; } } Also, there is some endian-related confusion, at least for signatures. Sometimes they are stored in an array of bytes, whose order may be always the same, regardless of host or target. However, some were apparently hand-reordered between pm3 and cm3, and the reorderings are neither strictly endianness nor consistent for different types. I seem to recall something like swapping 32-bit segments, but not bytes within them, or the other way around, or something, for some but not all types. I had to deal with both versions of the orderings in pickles and maybe in m3gdb. Here is an excerpt from Pickle2.m3: (* Adaptation to some builtin fingerprints that have different byte order in Pm3 and Cm3. *) TYPE FPA = ARRAY [0..7] OF BITS 8 FOR [0..255]; (* Give a short name to anonymous type of Fingerprint.T.byte. *) CONST NULL_uid = 16_48ec756e; CONST pm3_NULL_Fp = FPA {16_24,16_80,16_00,16_00,16_6c,16_6c,16_75,16_6e}; CONST cm3_NULL_Fp = FPA {16_6e,16_75,16_6c,16_6c,16_00,16_00,16_80,16_24}; CONST ROOT_uid = 16_9d8fb489; CONST pm3_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_65,16_86,16_ad,16_41}; CONST cm3_ROOT_Fp = FPA {16_41,16_ad,16_86,16_65,16_c8,16_19,16_09,16_f8}; CONST UNTRACED_ROOT_uid = 16_898ea789; CONST pm3_UNTRACED_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_71,16_87,16_be,16_41}; CONST cm3_UNTRACED_ROOT_Fp = FPA {16_41,16_be,16_87,16_71,16_c8,16_19,16_09,16_f8}; (* Can the following two occur in a pickle? Maybe if somebody registered a special for them? *) CONST ADDRESS_uid = 16_08402063; CONST pm3_ADDRESS_Fp = FPA {16_91,16_21,16_8a,16_62,16_f2,16_01,16_ca,16_6a}; CONST cm3_ADDRESS_Fp = FPA {16_f2,16_01,16_ca,16_6a,16_91,16_21,16_8a,16_62}; CONST REFANY_uid = 16_1c1c45e6; CONST pm3_REFANY_Fp = FPA {16_65,16_72,16_24,16_80,16_79,16_6e,16_61,16_66}; CONST cm3_REFANY_Fp = FPA {16_66,16_61,16_6e,16_79,16_80,16_24,16_72,16_65}; Hope this is a step forward in the way of helpful information > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From jay.krell at cornell.edu Thu Aug 16 20:43:02 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 16 Aug 2012 14:43:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> References: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Message-ID: Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised". I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient. But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis. I have to figure out something for nested functions... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: > Yes, I've thought so for a long time. It would also more easily enable llvm. > > Sent from my iPhone > > On Aug 16, 2012, at 10:21, Jay K wrote: > >> Should m3cg provide enough information for a backend to generate idiomatic C? >> (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) >> >> >> Should we make it so? >> >> >> Or be pragmatic and see if anyone gets to that point? >> >> >> But, look at this another way. >> Let's say we are keeping the gcc backend. >> >> >> Isn't it reasonable to have a better experience with stock gdb? >> >> >> What should m3cg look like then? >> >> >> Matching up m3front to gcc turns out to be "wierd". >> As does having a backend generate "C". >> >> >> In particular, "wierd" because there is a "level mismatch". >> >> >> m3cg presents a fairly low level view of the program. >> It does layout. Global variables are stuffed into what you might call a "struct", with >> no assigned field names. Field references are done by adding to addresses and casting. >> >> >> Too low level to provide a "good" gcc tree representation or to generate "normal" C. >> >> >> One might be able to, by somewhat extraordinary means, make due. >> That is, specifically one could deduce field references from >> offsets/sizes. But maybe it is reasonable for load/store >> to include fields? Maybe in addition to what it provides? >> >> >> As well, it appears to me, that >> >> >> given TYPE Enum = {One, Two, Three}; >> >> the m3cg is like: >> >> declare enum typeidblah >> declare enum_elt One >> declare enum_elt Two >> declare enum_elt Three >> declare_typename typeidblah Enum >> >> >> One kind of instead wants more like: >> >> >> declare enum typeidblah Enum >> declare enum_elt One => rename it Enum_One >> declare enum_elt Two "" >> declare enum_elt Three "" >> >> >> However I understand that {One, Two, Three} exists >> as anonymous type independent of the name "Enum". >> >> >> One could just as well have: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = {One, Two, Three}; >> >> >> Enum1 and Enum2 probably have the same typeid, and are just >> two typenames for the same type. >> >> >> likewise: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = Enum1; >> >> >> but, pragmatically, in the interest of generating better C, >> can we pass a name along with declare_enum? >> >> I ask somewhat rhetorically. I realize there is the answer: >> enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >> typedef enum Mtypeid Enum1; >> >> >> Also, enum variables I believe end up as just UINT8, 16, or 32. >> Loads of enum values I believe end up as just loads of integers. >> Can we pass along optional enum names with declare_local/declare_param? >> And optional enum names with load_int? >> Or add a separate load_enum call? >> >> >> Really, I understand that the current interface can be pressed to do >> pretty adequate things. I can infer field references. The way enums work >> isn't too bad. >> >> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 23:13:55 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 22:13:55 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345151635.43876.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: Jay, even in RISC like G. Nelson mentioned there was this problem of big executables, and I happen to believe this doesn't change things too much for the better, so be careful with summing all kind of stuff will go on OK, there could be the side effects that we don't like too much. In the other side SPIN M3 Vortex compiler was a DEC-SRC modified front-end for that back-end to a Java-like subset backend, I don't know how well it was that, but at least they have done that already to the front-end so is less work to do. I don't care if you redefine the language to admittedly drop some unwanted changes but with some purpose of simplifying things, this was the best effort they did so we need to be aware of that. I generally don't take to heart this part of the system, but we need to respect the existence of this backend if somebody has a new better and brave CG that's better but to use your own language to do this I don't like the idea of several backends and different languages. Thanks in advance --- El jue, 16/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Antony Hosking" CC: "m3devel" , "Jay K" Fecha: jueves, 16 de agosto, 2012 13:43 Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised".? I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient.?But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis.I have to figure out something for nested functions... ?- Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: Yes, I've thought so for a long time. It would also more easily enable llvm.? Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 17 17:16:21 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 17 Aug 2012 11:16:21 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502D2D75.8010702@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> <502D2D75.8010702@lcwb.coop> Message-ID: <20120817151621.GB19487@topoi.pooq.com> On Thu, Aug 16, 2012 at 12:27:17PM -0500, Rodney M. Bates wrote: > > > On 08/15/2012 07:55 PM, Hendrik Boom wrote: > >On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > >> > >> > >>On 08/14/2012 10:04 PM, Jay K wrote: > >>> > >>>Heck, even if these weren't global, is it that unreasonble, > >>>from the programmer's point of view, for the language/compiler > >>>to do some pointer escape analysis and let me take the address > >>>of a local, as long as I don't store it somewhere that outlives > >>>the local? > >>> > >> > >>This is ultimately an undecidable problem and even conservative > >>approximations of reasonable sophistication are far too involved > >>for a language to require of every compiler. > > > >Not to mention the obscurityof the language definition. > > > > Yes, yes, yes!!! > > >But isn't there something like that with the addresses of top-level > >proocedures? the ones whose environments are completely static? > >Or am I thinking of another type-safe language? Or am I thinking of > >assigning the procedures themselves? > > Yes. If you can assign a procedure (or pointer to function, which is > equivalent, for this discussion) to a variable, then there are ways to > call the procedure when its environment does not exist, i.e, one or > more scopes of variables that the procedure is declared inside of and > can refer to. The outermost scope always exists, so we need three > nested scopes to get an example. > > MODULE Outer > > ; PROCEDURE Middle ( MiddleFormal : INTEGER ) > > = PROCEDURE Inner ( ) : INTEGER > > = BEGIN > RETURN MiddleFormal > END > > ; BEGIN (* Middle *) > ProcVar := Inner (* Statically illegal because Inner is not top-level. *) > END (* Middle > > ; VAR ProcVar : PROCEDURE ( ) : INTEGER > ; VAR X : INTEGER > > ; BEGIN (* Outer *) > Middle ( 7 ) > ; X := ProcVar ( ) > END Outer > . > > If the assignment ProcVar := Inner were legal, the call ProcVar ( ) > goes to Inner, which refers to MiddleFormal of Middle, but there is no > active invocation of Middle, and MiddleFormal doesn't exist. > > Pascal (if I remember all this right) allows procedures to be passed > as parameters, but not assigned to variables. The procedure constant > in question can only be named where it has an environment. If all you > can do with it is pass it, any use of it can only be deeper inside the > dynamic call chain, where its environment continues to exist. > > Modula-2 allows procedures to be either passed or assigned, but in > either case, only a top-level procedure, which always has an > environment, can be given. > > Both passing and assigning procedures have reasonable uses, both for > variations on the callback theme. > > Modula-3 supports both by allowing any procedure to be passed but only > a top-level one to be assigned. Enforcing the latter restriction > sometimes means a runtime check at the point of an assignment, if the > RHS of the assignment is a procedure formal parameter, which could > dynamically be top-level or not. This is probably not a > high-frequency situation. Assigning a procedure constant and passing > any procedure can be fully checked at compile time. > > I think it is possible to define a weaker, yet safe rule that > postpones runtime environment existence checking until the point of a > call on a procedure variable, but I don't recall seeing this in any > language. Maybe Algol68 or something. Algol 68 restricted assignment. The 'scope' of the procedure had to be at most as deeply nested as the 'scope' of the variable. This was a general restriction that applied to everything, though, not just procedures. This meant you could assign nested procedures to more nested variables. The 'scope' of a procedure was determined by its most local global variable. But scope checking led to heavy overhead on just about anything having to do with pointers, so it tended to be rarely inplemented. As on implementer said, this isn't the kind of mistake programmers make, anyway. The CDC Cyber implementation performed the check when called. Each scope (or rather, each scope for with this situation could arise) caused a small object to be allocated on the heap on scope entry. The stack frame would point to this object, and the object would point back to the stack frame. A procedure would contain a pointer to this object for the stak frame contining its most local globals.. WHen the procedure was called, it would check that this object indeed pointed to the place on the stack frame where there would be a pointer back to the object. Still, the restriction eliminated the possibility of writing a function-composition function. The only way I see of making this more liberal is to perform escape analysis on procedures, and subsequently choosing a few blocks whose stack frame would be allocated on the heap. You could apply run-time tests to prevent you from jumping into stack frames that have already been exited (of course in Modula 3, there aren't any GOTOs. It turns out that the analysis isn't all that difficult, and that for normal programs, almost no frames get allocated on the heap, hence little run-time overhead. The mechanism could be useful for callbacks. This is is the kind of analysis every decent Scheme compiler does to avoid accomodating the full force of conitinuation passing where it is not needed. -- hendrik > > In gcc-enhanced C, which allows nested functions, in addition to both > assigning and passing (pointers to) functions, you are on your own to > avoid calling something that lacks an environment. The one help the > language gives is to say in the reference, "all hell will break loose" > if you get this wrong. Does gcc-enhanced C allow you to put stack frames on the heap? -- hendrik From rodney_bates at lcwb.coop Fri Aug 17 20:51:12 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 13:51:12 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502E92A0.5080809@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } Hmm, this is not quite what I had in mind, but it's close. I like to code things like this with a pointer to the interesting node _and_ one that lags one node behind, in case list surgery is needed. Also, a more complete example would refer to one or more data fields of a node too. But I think I can illustrate my intentions with your example. > > This can be done prefectly type-safe in languages that allow > > REF REF list Plus, an ADR/& operation. In the case of C's &, it's type at least remembers the type of the object whose address is taken. Modula-3's ADR does not, so would require a LOOPHOLE as well. But no matter, because an operator that can take the address of anything other than a heap object (or maybe a component of one?) is unsafe. > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Fri Aug 17 21:12:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:12:35 -0500 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <502E97A3.2090804@lcwb.coop> On 08/16/2012 09:21 AM, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. In the stabs debug info given to (m3)gdb, they do have field names, with the original source code field name embedded within, plus a character-encoded uid and some flags, etc. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > This is the more reasonable representation, in view of the fact that any type expression is itself anonymous, as you note below, while giving it a type name is a separate matter, done outside the type definition. > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". Yes. > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > Yes. > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > This business of anonymous types is pervasive, not just for enumeration types and not just in Modula-3. It shows up all over the place in StubGen, which has lots of code obviously intended to handle anonymous object types and, not so obviously, broken, or at least unfinished. I guess nobody actually does it. Having once been inside that code, I would not define an anonymous type I wanted to have involved in anyway with network objects. Anonymous types are usually not particularly good coding style, and I almost always regret defining one and end up giving it a name after all, as I want to use it in multiple places. Modula-3's structural type equality rules perhaps encourage it more than so-called name equality. C encourages it too for the declarator types and by requiring manual construction of pass-by-reference by the programmer, instead of parameter modes. Something similar for the second-class treatment of arrays. > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > I don't remember the details, but m3gdb gets enumeration value names, displays them, and recognizes them in commands. They had to go through parse.c to get to stabs. > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > I know this is a broken record, but the fact that debug info comes off the side of parse.c means it can't reflect optimizations/transformations done later by the gcc backend. E.g. tree-nested.c. > - Jay From rodney_bates at lcwb.coop Fri Aug 17 21:28:53 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:28:53 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502E9B75.8040100@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive definitions and declarations, your M3 example being one. Modula-3's rule that all declarations in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, along with separate rules explicitly disallowing certain forms of recursive declarations that would be troublesome. E.g., deleting both occurrences of "REF" above would give records of infinite size. You might say the language uses "opt-out" for certain recursive declarations. Most languages use an opt-in strategy. The default rules is declare-before-use. For select kinds of declarations, there is some kind of "forward" declaration that declares the name and only some of its properties. After that, the name then can be used in limited ways (most interestingly, in other declarations). Eventually, there has to be a complete declaration that give the name again and supplies the remaining properties. After that it can be used in all legal ways. A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- use strategy with a system of forward declarations. My impression is that gcc is this way, but poorly documented. Will it give enough hints to just look at the front end output for selected recursive declaration sets like this example? It would at least give something known to be acceptable to gdb. Or is gdb changing the rules with new releases? > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From rodney_bates at lcwb.coop Sat Aug 18 19:51:11 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 18 Aug 2012 12:51:11 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: <502FD60F.7050002@lcwb.coop> On 08/15/2012 05:07 PM, Jay K wrote: > > If fact, if you are accustomed to thinking in type-safe terms > > > I do think in fairly type-terms. > In C++ I know to use templates, for example. > There is a matter of lifetime-safety though. > I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. > > > Consider C++ code like this, where I want to do some things > "the same" to a bunch of variables, and some things "specific". > > > struct Person_t { const char* name; }; > > void F() > { > Person_t jay {"jay"}; > Person_t randy = {"randy"}; > Person_t tony = {"tony"}; > > // print all their names > Person_t* people[] = {&jay, &randy, &tony}; > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i]->name); > > DoSomethingSpecificToJay(&jay); > DoSomethingSpecificToRandy(&randy); > DoSomethingSpecificToTony(&tony); > > } > In Modula-3, you can code like this too, but you have to declare it UNSAFE and use unsafe constructs, just like you have to use in C, but don't have to admit explicitly . > I do NOT want to do this: > > void F() > { > Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; > > > // print all their names > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i].name); > > DoSomethingSpecificToJay(&people[0]); > DoSomethingSpecificToRandy(&people[1]); > DoSomethingSpecificToTony(&people[2]); > > } > > I understand there is: > enum People { jay, randy, tony }; > ... > DoSomethingSpecificToJay(&people[jay]); > DoSomethingSpecificToRandy(&people[randy]); > DoSomethingSpecificToTony(&people[tony]); > ... > > > but I'm still not thrilled about that. > > I ended up without anything "specific" except initialization..so no > really need to hang a name/variable on the data, and the > initialization is fairly specific, so the resulting code is merely: > > > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); > > EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); > > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); > > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); > > ... > > you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems > in the m3cg interface...these types need to be defined via function calls into the backend, > not hardcoded..and even if they are...the order of types arriving at the backend isn't always > ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... > I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) > > > - Jay > > > > From: jay.krell at cornell.edu > > Date: Wed, 15 Aug 2012 12:56:09 -0400 > > To: hosking at cs.purdue.edu > > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] reference to globals in globals? > > > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > > > - Jay (briefly/pocket-sized-computer-aka-phone) > > > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > > > Jay, > > > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > > > >> > > >> > > >> On 08/14/2012 10:04 PM, Jay K wrote: > > >>> Isn't it safe to take the address of a global? > > >>> > > >> > > >> Do you mean can't you use the ADR function in safe code > > >> if you apply it only to a global variable? The answer > > >> to that is no. The ADR function is illegal altogether in > > >> safe code. > > >> > > >> As to why, I can only speculate, but see below. I suspect > > >> even in this case, it is not as simple as it seems. > > >> > > >>> > > >>> I have something like this: > > >>> > > >>> > > >>> CONST UID_INTEGER = 1234; > > >>> CONST UID_FLOAT = 4567; > > >>> ... several more ... > > >>> > > >>> > > >>> TYPE CType = OBJECT .. END; > > >>> > > >>> > > >>> VAR t_int: CType := ...; > > >>> VAR t_float: CType := ...; > > >>> ... several more ... > > >>> > > >>> > > >>> MapTypeIdToType(UID_INTEGER, t_int); > > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > > >>> ... several more ... > > >>> > > >>> > > >>> but what I really want is more like: > > >>> > > >>> > > >>> TYPE RECORD = BuiltinUid_t = > > >>> typeid: INTEGER; > > >>> ctype: REF CType; > > >> > > >> ^UNTRACED REF? If it were just REF, that would imply that > > >> your global variable (the pointer it contains) is a heap object, that > > >> it has heap allocator/GC overhead data attached to it, and that the GC > > >> should trace it, none of which is true. > > >> > > >> > > >>> END; > > >>> > > >>> > > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > > >>> BuiltinUids{UID_INTEGER, &t_int}, > > >>> BuiltinUids{UID_FLOAT, &t_float}, > > >> > > >> ADR instead of &? If so, you are still not there, because ADR > > >> returns a value of type ADDRESS, i.e., an untraced reference to > > >> we-don't-know-what. Somewhere, you would also have to use a > > >> LOOPHOLE to get it to UNTRACED REF CType. > > >> > > >>> ... several more ... > > >>> }; > > >>> > > >>> > > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > > >>> END; > > >>> > > >> > > >> I don't know what the signature of MapTypeIdToType is, but above, > > >> you are passing a variable of object type to its 2nd parameter, > > >> (which contains a traced reference to the actual heap object). > > >> But here, you pass the _address_ of the above. Inconsistent > > >> number of levels of indirection. A static safe language is > > >> much more likely to help with things like this. > > >> > > >> Maybe you just want to say > > >> > > >> TYPE RECORD = BuiltinUid_t = > > >> typeid: INTEGER; > > >> ctype: CType; > > >> > > >> and > > >> > > >> BuiltinUids{UID_INTEGER, t_int}? > > >> > > >> This would be equivalent to your first way, and doesn't require any > > >> unsafe coding at all. > > >> > > >> Or, you could do away with global variable t_int altogether and > > >> just initialize directly into BuiltinUids[..].ctype with whatever > > >> expression you used to initialize t_int. It looks like your array > > >> makes the t_int and cousins redundant. > > >> > > >>> > > >>> Heck, even if these weren't global, is it that unreasonble, > > >>> from the programmer's point of view, for the language/compiler > > >>> to do some pointer escape analysis and let me take the address > > >>> of a local, as long as I don't store it somewhere that outlives > > >>> the local? > > >>> > > >> > > >> This is ultimately an undecidable problem and even conservative > > >> approximations of reasonable sophistication are far too involved > > >> for a language to require of every compiler. > > >> > > >>> > > >>> You can see this particular pattern currently in > > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > >>> > > >>> > > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > > >>> where I encounter this. > > >>> > > >>> > > >>> Working in safe languages can be frustrating... > > >>> > > >> > > >> It's just an instant/deferred gratification thing. Safe languages often > > >> make you stop and fix it before you run it. Unsafe languages let you naively > > >> forge ahead to the next step, where the bug is likely to be *much* harder to > > >> diagnose, assuming you even have enough test cases to notice it at all during > > >> development. Your code here is a good example. > > >> > > >> Of course, safe languages occasionally make you unnecessarily write a bit more > > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > > >> linked list example. In my personal experience, these times are at least > > >> one order of magnitude less frequent than the times safe languages reduce > > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > > >> first place. > > >> > > >> You're making it too difficult. > > >> > > >>> > > >>> Thank you, > > >>> - Jay > > >> > > > From jay.krell at cornell.edu Sun Aug 19 09:11:30 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:11:30 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? Message-ID: I propose the following changed to m3front, TInt, TFloat. It does kind of "infact" the front/middle end with the "intent" of a specific backend. When a TInt or TFloat is initialized from text, that text is remembered. When such a TInt/TFloat is added/subtracted/multiplied/divided by another, the obvious text is built up in the target. Initialization of TInt with constants could/should also maintain the textual representation. The implication is multi part: 1) init_float in a C-or-such backend can just output the textual expression. No need to worry about roundtripping it. 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" (preceded by a typedef for INTEGER in the larger context)..thereby factoring out the target from the generated C. 3) This provides some hypothetical portability when host/target don't use the "same" sort of floating point. Ignoring rounding details however. And TFloat does seem to round correctly at compile-time, assuming host/target are about the same -- and they all are. (What does the language definition say here anyway? I'll have to check..) Reasonable? For now I'll probably punt. I can just use TFloat.ToChars and munge it slightly. But this is probably a good direction. In the short term I'm trying hard to make do w/o m3front/m3middle changes. You know..implementing another backend, you tend to visit more of the system, gradually understand it..think of what you how one might prefer it to be..esp. for the task at hand. :) Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be?If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 19 09:14:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:14:23 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <502E9B75.8040100@lcwb.coop> References: , , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu>, , <502E9B75.8040100@lcwb.coop> Message-ID: Thank for the pointers to m3gdb -- it kind of confirms my belief that the system isn't great.Even if there is a bootstrapping problem, it seems wrong for each backend and m3gdb to pay for it.Just m3front/m3front should have this hardcoded information. There is an m3front -emit option to ... i think provide the information..but it fails.I suspect it used and works when building just m3core.I have to look more. For now I'm punting on paying any attention to type information in the C-backend-in-progress.Just like the other backends ignore it all also.(ok..gcc uses it in the wierd way for m3gdb...) - Jay > Date: Fri, 17 Aug 2012 14:28:53 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > > > On 08/15/2012 09:07 PM, Jay K wrote: > > > There is a bootstrap issue here isn?t there? > > > > I am not sure. I don't understand the system enough. :( > > > > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > > The compiler does for example construct the type BOOLEAN out of lower level pieces. > > Can it not then proceed to compute its typeid, and inform the backend about it? > > > > > > Other examples appear to include...wait.where did I get this list? > > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > > I built up the following list: > > > > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > > UID_LONGINT = 16_05562176; (* LONGINT *) > > UID_WORD = 16_97E237E2; (* CARDINAL *) > > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > > UID_REEL = 16_48E16572; (* REAL *) > > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > > UID_WIDECHAR = 16_88F439FC; > > UID_MUTEX = 16_1541F475; (* MUTEX *) > > UID_TEXT = 16_50F86574; (* TEXT *) > > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > > UID_ROOT = 16_9D8FB489; (* ROOT *) > > UID_REFANY = 16_1C1C45E6; (* REFANY *) > > UID_ADDR = 16_08402063; (* ADDRESS *) > > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_NULL = 16_48EC756E; (* NULL *) > > > > > > Surely these aren't all fundamental?? > > > > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > > > > TYPE Record1 = RECORD REF Record2 END; > > TYPE Record2 = RECORD REF Record1 END; > > > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > > > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive > definitions and declarations, your M3 example being one. Modula-3's rule that all declarations > in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, > along with separate rules explicitly disallowing certain forms of recursive declarations > that would be troublesome. E.g., deleting both occurrences of "REF" above would give > records of infinite size. You might say the language uses "opt-out" for certain recursive > declarations. > > Most languages use an opt-in strategy. The default rules is declare-before-use. For select > kinds of declarations, there is some kind of "forward" declaration that declares the name > and only some of its properties. After that, the name then can be used in limited ways (most > interestingly, in other declarations). Eventually, there has to be a complete declaration that > give the name again and supplies the remaining properties. After that it can be used in all > legal ways. > > A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- > use strategy with a system of forward declarations. My impression is that gcc is this way, > but poorly documented. > > Will it give enough hints to just look at the front end output for selected recursive declaration > sets like this example? It would at least give something known to be acceptable to gdb. Or > is gdb changing the rules with new releases? > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > From: hosking at cs.purdue.edu > > Date: Wed, 15 Aug 2012 19:58:23 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3 backends and builtin types? > > > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > > > Something seems off to me in the current implementation. > > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > > I hardcoded knowledge of them in parse.c and M3C.m3. > > That seems wrong. > > > > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > > > > Shouldn't m3front inform the backend via m3cg of these types? > > It is doable using the existing interfaces? > > > > > > More so,RTBuiltin.mx ought not exist, right? > > Whatever data it contains should be built up like any other type data? > > Part of the same problem? > > > > > > - Jay > > > > > > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > > 305 N. University Street | West Lafayette | IN 47907 | USA > > Mobile+1 765 427 5484 > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 19 17:45:00 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 19 Aug 2012 11:45:00 -0400 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <20120819154500.GA17607@topoi.pooq.com> On Sun, Aug 19, 2012 at 07:11:30AM +0000, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. How many digits are you planning to keep for 1.0/3.0 ? -- hendrik From rodney_bates at lcwb.coop Sun Aug 19 23:48:27 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 19 Aug 2012 16:48:27 -0500 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <50315F2B.2060300@lcwb.coop> On 08/19/2012 02:11 AM, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > The implication is multi part: > > > 1) init_float in a C-or-such backend can just output the textual expression. > No need to worry about roundtripping it. > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > out the target from the generated C. > > > 3) This provides some hypothetical portability when host/target > don't use the "same" sort of floating point. Ignoring rounding details however. > And TFloat does seem to round correctly at compile-time, assuming host/target > are about the same -- and they all are. > (What does the language definition say here anyway? I'll have to check..) > As far as I can find, the language says nothing about this. However, I feel *extremely* strongly that arithmetic done at compile time should be *exactly* the same as on the target machine, including what happens in exceptional cases, e.g., overflows, NaNs, etc. I know this can be hard to get right, and it raises hard questions about division of work between front/back. Even harder when there are multiple back ends, especially ones you don't control. But I consider it a serious compiler bug if CT and RT arithmetic differ. This might be something that could be postponed for more urgent things, but the design at least should anticipate it. I have no problems with the principal of preserving/generating textual representations. However, if you really implement target arithmetic somewhere, it might be easier to convert back and forth than to do faithful target arithmetic on character strings. If that is what you are referring to as {not} roundtripping. > > Reasonable? > > > For now I'll probably punt. > I can just use TFloat.ToChars and munge it slightly. > But this is probably a good direction. > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > You know..implementing another backend, you tend to visit more of the system, > gradually understand it..think of what you how one might prefer it to be..esp. > for the task at hand. :) > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > - Jay > > From jay.krell at cornell.edu Mon Aug 20 00:08:26 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 22:08:26 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: <50315F2B.2060300@lcwb.coop> References: , <50315F2B.2060300@lcwb.coop> Message-ID: > However, if you really implement target arithmetic somewhere, I don't. I just want to print out constants or expressions for a C or C++ compiler to deal with. My initial goal is a C-generating backend that "work". The code will be target-dependent and not very debuggable (like what stock debuggers do today with Modula-3). Longer term, I want the code to be target-endian-indepenent, target-wordsize-independent, target-jmpbuf-independent, and possibly floating-point-representation-independent -- completely target-independent. Floating-point-representation independence doesn't gain anything -- all hosts and targets use the same representation, and I think the current system depends on that anyway (see TFloat). Floating-point-representation independence is "difficult" for "most compilers", but if generating C/C++, I think it is easy. target-wordsize-independence I believe shall be gained by my proposal here. target-endian-indepencence I believe shall be gained by small changes in m3front and the existing backends -- m3front only checks endianness in like one or two places. target-jmpbuf-size-independence I believe shall be gained by roughly the change I made in late 2010, but w/o causing loops to allocate stack for every try "invocation". Or, well, there is likely another way -- some indication to the backend that the variable is a jumpbuf and then outputing "jmp_buf jb" in the C backend..it should be easy enough..add declare_jmpbuf that has the same signature as declare_local, or add a boolean to declare_local. That is more efficient than the alloca(Csetjmp_jmpbuf_size) approach, but doesn't achieve the same gain for m3middle -- that is, even when targeting m3cc, m3front/m3middle ought not have this information -- that would cut out yet one more target-dependence in cm3/m3front/m3middle. Anyway, I'm not to this point yet. I have a fair amount more to do just for target-dependent C to be generated and work. I have made progress, but there is more to do. - Jay > Date: Sun, 19 Aug 2012 16:48:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? > > > > On 08/19/2012 02:11 AM, Jay K wrote: > > I propose the following changed to m3front, TInt, TFloat. > > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > > > > When a TInt or TFloat is initialized from text, that text is remembered. > > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > > the obvious text is built up in the target. > > > > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > > > > The implication is multi part: > > > > > > 1) init_float in a C-or-such backend can just output the textual expression. > > No need to worry about roundtripping it. > > > > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > > out the target from the generated C. > > > > > > 3) This provides some hypothetical portability when host/target > > don't use the "same" sort of floating point. Ignoring rounding details however. > > And TFloat does seem to round correctly at compile-time, assuming host/target > > are about the same -- and they all are. > > (What does the language definition say here anyway? I'll have to check..) > > > > As far as I can find, the language says nothing about this. > > However, I feel *extremely* strongly that arithmetic done at > compile time should be *exactly* the same as on the target machine, > including what happens in exceptional cases, e.g., overflows, > NaNs, etc. > > I know this can be hard to get right, and it raises hard questions > about division of work between front/back. Even harder when there > are multiple back ends, especially ones you don't control. > But I consider it a serious compiler bug if CT and RT arithmetic > differ. This might be something that could be postponed for more > urgent things, but the design at least should anticipate it. > > I have no problems with the principal of preserving/generating > textual representations. However, if you really implement target > arithmetic somewhere, it might be easier to convert back and forth > than to do faithful target arithmetic on character strings. If > that is what you are referring to as {not} roundtripping. > > > > > Reasonable? > > > > > > For now I'll probably punt. > > I can just use TFloat.ToChars and munge it slightly. > > But this is probably a good direction. > > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > > > > You know..implementing another backend, you tend to visit more of the system, > > gradually understand it..think of what you how one might prefer it to be..esp. > > for the task at hand. :) > > > > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > > > > - Jay > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Aug 21 13:18:48 2012 From: dmuysers at hotmail.com (Dirk Muysers) Date: Tue, 21 Aug 2012 13:18:48 +0200 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: *** A warning *** Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: Code generation is my business :-) Comments on a few options: a.. CLR: a.. Pro: industrial support b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter c.. Con: Only Windows platform is really prime-time quality b.. LLVM: a.. Pro: enthusiastic user community with charismatic leader b.. Pro: serious backing from Apple c.. Pro: many interesting performance improvements d.. Con: somewhat complex interface e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface c.. C-- a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code b.. Pro: design is reasonably mature and reasonably clean c.. Pro: supports accurate garbage collection d.. Pro: most users report it is very easy to use e.. Con: very small development team f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) g.. Con: does not ship with a garbage collector h.. Con: project has no future d.. C as target language a.. Pro: looks easy b.. Con: nearly impossible to get decent performance c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project. From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 14:33:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 13:33:27 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345552407.75520.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: It might need some pre-processor but C is supposedly superior in performance than C# and Java by far, I mean we are not talking about using C, we are talking about generating object code from UNSAFE code, but it shouldn't be that bad in terms of assembly level compile and link time process compared to those systems, if not then why every body wants their software be written in (e.g us) Modula-3, but compiled in a C environment it must be something about engineering causes of who wrote that thing first, and C is much older, so there is better support for that UNSAFE level code (instead look C3 and Java, lame), and that's for me more important in this matter. Probably C is referred as a high-level assembly (like DEC-SRC first versions), instead of a low-level assembly language like here we are trying to do. In fact if we were able to generate C from M3CG we can gain better portability, for instance my project of compare cross-boots would be more easily than by itself now. Thanks in advance --- El mar, 21/8/12, Dirk Muysers escribi?: De: Dirk Muysers Asunto: Re: [M3devel] higher level m3cg? Para: "Jay K" , "m3devel" Fecha: martes, 21 de agosto, 2012 06:18 *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 21 16:10:02 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Tue, 21 Aug 2012 10:10:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr. Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > CLR: > > Pro: industrial support > Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > Con: Only Windows platform is really prime-time quality > LLVM: > > Pro: enthusiastic user community with charismatic leader > Pro: serious backing from Apple > Pro: many interesting performance improvements > Con: somewhat complex interface > Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface > C-- > > Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > Pro: design is reasonably mature and reasonably clean > Pro: supports accurate garbage collection > Pro: most users report it is very easy to use > Con: very small development team > Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > Con: does not ship with a garbage collector > Con: project has no future > C as target language > > Pro: looks easy > Con: nearly impossible to get decent performance > Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 16:53:37 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 15:53:37 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <1345560817.79160.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: target dependency is essential in this process, so a good backend must target an quintessential machine, so I don't agree a given machine will give us lot of optimization without lot of work: http://irclog.perlgeek.de/parrot/2010-07-25#i_2605776 M3CG can be converted to a virtual machine as an emulator of an automaton, but studying that automaton complexity will give us the real facts (instead on believing stories), I don't want to fight wars among languages, but by practice we now know that C is far easier to code low level stuff than everything else, so .. if we want a radical change let's study why we do it first. Thanks in advance --- El mar, 21/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: "m3devel" , "Jay K" Fecha: martes, 21 de agosto, 2012 09:10 I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr.? Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Tue Aug 21 20:14:24 2012 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Tue, 21 Aug 2012 20:14:24 +0200 (CEST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: On Tue, 21 Aug 2012, Antony Hosking wrote: > *** A warning *** > Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: > ? > > Code generation is my business :-) > > Comments on a few options: > > * > > CLR: > > + Pro: industrial support > + Con: you have to buy into their type system pretty much completely; depending on what you want > to do with types, this may not matter > + Con: Only Windows platform is really prime-time quality > * > > LLVM: > > + Pro: enthusiastic user community with charismatic leader > + Pro: serious backing from Apple > + Pro: many interesting performance improvements > + Con: somewhat complex interface > + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > to be plugged by adding to the complexity of the interface Additional Con: They introduce bugs from time to time and then they do not fix them ... > * > > C-- > > + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > your C-- code LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > + Pro: design is reasonably mature and reasonably clean > + Pro: supports accurate garbage collection > + Pro: most users report it is very easy to use > + Con: very small development team > + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > + Con: does not ship with a garbage collector > + Con: project has no future > * > > C as target language > > + Pro: looks easy > + Con: nearly impossible to get decent performance > + Con: will drive you nuts in the long run; ask the long line of people who have tried to > compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > of these people gave up and built their own native code generator. > > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > and expected longevity, I'd probably recommend LLVM. Despite my 'con' point I would recommend LLVM, too. From jay.krell at cornell.edu Tue Aug 21 21:26:49 2012 From: jay.krell at cornell.edu (Jay) Date: Tue, 21 Aug 2012 12:26:49 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >> *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >> >> Code generation is my business :-) >> Comments on a few options: >> >> * >> CLR: >> >> + Pro: industrial support >> + Con: you have to buy into their type system pretty much completely; depending on what you want >> to do with types, this may not matter >> + Con: Only Windows platform is really prime-time quality >> * >> LLVM: >> >> + Pro: enthusiastic user community with charismatic leader >> + Pro: serious backing from Apple >> + Pro: many interesting performance improvements >> + Con: somewhat complex interface >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >> to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >> C-- >> >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >> your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >> + Pro: design is reasonably mature and reasonably clean >> + Pro: supports accurate garbage collection >> + Pro: most users report it is very easy to use >> + Con: very small development team >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >> + Con: does not ship with a garbage collector >> + Con: project has no future >> * >> C as target language >> >> + Pro: looks easy >> + Con: nearly impossible to get decent performance >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >> of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > From dabenavidesd at yahoo.es Tue Aug 21 23:58:52 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 22:58:52 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: <1345586332.94616.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: at the core of this issue, and if you want to ellucidate more, please C Ramsey, Lambda-RTL, and Necula Lambda-CIL respectively: http://terohasu.net/desclang/Description_languages.html#LAMBDA-RTL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.72.4008 Baby Modula-3 permits axiomatic reasoning over objects so it should be possible to reason over given program an its execution in a system of given semantics Thanks in advance --- El mar, 21/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Henning Thielemann" CC: "m3devel" , "JayK" Fecha: martes, 21 de agosto, 2012 14:26 I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >>? ? ? *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >>? >> Code generation is my business :-) >> Comments on a few options: >> >> * >>? ? CLR: >> >>? ???+? Pro: industrial support >>? ???+? Con: you have to buy into their type system pretty much completely; depending on what you want >>? ? ? ? to do with types, this may not matter >>? ???+? Con: Only Windows platform is really prime-time quality >> * >>? ? LLVM: >> >>? ???+? Pro: enthusiastic user community with charismatic leader >>? ???+? Pro: serious backing from Apple >>? ???+? Pro: many interesting performance improvements >>? ???+? Con: somewhat complex interface >>? ???+? Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >>? ? ? ? to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >>? ? C-- >> >>? ???+? Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >>? ? ? ? your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >>? ???+? Pro: design is reasonably mature and reasonably clean >>? ???+? Pro: supports accurate garbage collection >>? ???+? Pro: most users report it is very easy to use >>? ???+? Con: very small development team >>? ???+? Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >>? ???+? Con: does not ship with a garbage collector >>? ???+? Con: project has no future >> * >>? ? C as target language >> >>? ???+? Pro: looks easy >>? ???+? Con: nearly impossible to get decent performance >>? ???+? Con: will drive you nuts in the long run; ask the long line of people who have tried to >>? ? ? ? compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >>? ? ? ? of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 00:01:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Tue, 21 Aug 2012 22:01:23 +0000 Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> , <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: I find Ramsey's analysis rather incomplete. He mentions CLR has industrial strength support. But he fails to say that C also does, way more significantly so. And you can barely mention CLR w/o also mentioning Java, though CLR is more general. I know C has downsides. C, without C++, has the same poor exception handling. The performance of the compiler indeed will not be the best. The portability however will be tremendous. Basically every system I can think of except Windows Phone 7 and web browsers have a production quality C and C++ compiler. Examples of systems we don't support or support well today that a C/C++ backend would trivially enable, that are still manufactured, sold, updated: PPC{32,64}_AIX (still might need #ifdef in m3core) PPC64_{FREEBSD?,LINUX} ARM32_DARWIN (iPhone, iPad) MIPS64_{LINUX,OPENBSD} IA64_{VMS,HPUX,FREEBSD,LINUX} (still might need #ifdef in m3core due to the "two stacks"; I'm also worried that a 64bit integer closure marker is too small) ARM32_LINUX (including Android) MIPS32_LINUX (network routers!) ARM32_NT ARM64_? {PPC32,MIPS32,ARM32,I386}_CE plus some more obsolete systems: ALPHA{32,64}_VMS PPC64_DARWIN MIPS{32,64}_IRIX PA{32,64}_HPUX Granted, gcc/m3cc is *almost* as portable, and some of the above I have gotten far along in supporting -- really, given the gcc/m3cc/posix base, the biggest task to any port is getting the hardware up and running and with a decent C toolset. Plus that we would gain a "portable distribution format" that people could just tar xfz; cd; ./configure; make; make install Granted, I don't have much experience in any area here, and I don't yet have C working. LLVM will also take more time and learning. I probably won't put the time into it. I am hoping to get the C backend working, so that the cm3 system might be considered significantly more "done", so as new processors come along, no new porting will be needed, just as for the vast majority of code out there. LLVM supports many more architectures than listed below. Anyway..later.. - Jay > CC: hosking at cs.purdue.edu; dmuysers at hotmail.com; m3devel at elegosoft.com; jay.krell at cornell.edu > From: jay.krell at cornell.edu > Subject: Re: [M3devel] higher level m3cg? > Date: Tue, 21 Aug 2012 12:26:49 -0700 > To: lemming at henning-thielemann.de > > I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > > > > On Tue, 21 Aug 2012, Antony Hosking wrote: > > > >> *** A warning *** > >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > >> > >> Code generation is my business :-) > >> Comments on a few options: > >> > >> * > >> CLR: > >> > >> + Pro: industrial support > >> + Con: you have to buy into their type system pretty much completely; depending on what you want > >> to do with types, this may not matter > >> + Con: Only Windows platform is really prime-time quality > >> * > >> LLVM: > >> > >> + Pro: enthusiastic user community with charismatic leader > >> + Pro: serious backing from Apple > >> + Pro: many interesting performance improvements > >> + Con: somewhat complex interface > >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > >> to be plugged by adding to the complexity of the interface > > > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > > > > >> * > >> C-- > >> > >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > >> your C-- code > > > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > > > >> + Pro: design is reasonably mature and reasonably clean > >> + Pro: supports accurate garbage collection > >> + Pro: most users report it is very easy to use > >> + Con: very small development team > >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > >> + Con: does not ship with a garbage collector > >> + Con: project has no future > >> * > >> C as target language > >> > >> + Pro: looks easy > >> + Con: nearly impossible to get decent performance > >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to > >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > >> of these people gave up and built their own native code generator. > >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > >> and expected longevity, I'd probably recommend LLVM. > > > > Despite my 'con' point I would recommend LLVM, too. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 22 09:28:50 2012 From: wagner at elego.de (Olaf Wagner) Date: Wed, 22 Aug 2012 09:28:50 +0200 Subject: [M3devel] Fw: SW Bug #1080 Message-ID: <20120822092850.cef2e188.wagner@elego.de> I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system. I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From mika at async.caltech.edu Wed Aug 22 13:00:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 04:00:48 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822110048.903811A207D@async.async.caltech.edu> In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > > a.. CLR:=20 > > a.. Pro: industrial support=20 > b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader=20 > b.. Pro: serious backing from Apple=20 > c.. Pro: many interesting performance improvements=20 > d.. Con: somewhat complex interface=20 > e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 > b.. Pro: design is reasonably mature and reasonably clean=20 > c.. Pro: supports accurate garbage collection=20 > d.. Pro: most users report it is very easy to use=20 > e.. Con: very small development team=20 > f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 > g.. Con: does not ship with a garbage collector=20 > h.. Con: project has no future > d.. C as target language > > a.. Pro: looks easy=20 > b.. Con: nearly impossible to get decent performance=20 > c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 From jay.krell at cornell.edu Wed Aug 22 15:39:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:39:28 +0000 Subject: [M3devel] suggest fixed size integers in m3cg_binrd/binwr instead of compact encoding, for trivial decoding/looping? Message-ID: BinRd/BinWr "redesign". How about BinRd/BinWr is "redesigned" with the following in mind: instead of most compact encoding, instead use least-decoding-needed encoding, which is also least compact in particular, store all integers as native-endian 8 byte integers include at the start of the stream, like: UINT8 file_endian; UINT64 integer_map_offset; UINT64 integer_map_size; reader would mmap the file (mmap has its pluses and minuses). The data would be presumed to fit in address space -- even if it tends to be accessed sequentially, random access would be possible. if file_endian != host_endian, it would jump ahead to the positions indicated by integer_map_offset and swap them. writer would note all these offsets while writing If I made this change, then I'd be much more likely to just use this "pre-"existing" format as an in-memory format that I loop over multiple times. Given that these are temporary files that don't usually make it to disk, optimizing for size isn't clearly valuable. Given this change, walking the m3cg data multiple times would end up just being bumping a pointer through an array of variably-sized records, the sizes of which would be trivially looked up based on the first byte (with the exception of case_jump). This would likely obvious my ongoing little M3CG_MultiPass tedious work. Something I believe is desired/needed here for M3C.m3. This isn't meant to be a pointless excercise (though it is *somewhat* speculative). m3front/m3cg/m3cg_ops would be unchanged. The change would be in M3CG_BinRd.m3/M3CG_BinWr.m3/parse.c. Thoughts? I've omitted the obvious detail -- floats/doubles. There'd actually be an array of MultiByteValueMaps. They'd include a size. There'd be "4" for floats, and "8" for INTEGERs and doubles. And hypothetically 10 or 16 for "extended", but those don't really exist. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 15:36:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:36:28 +0000 Subject: [M3devel] m3cg "Proc" and "Var" Message-ID: I'm wondering if it might just be much easier if m3front assigns small incrementing integers for Var and Proc rather than getting pointers back from declare_procedure, declare_local, etc., and then passing them back to begin_procedure, load, etc. See, where I am at is I want to "playback" an m3cg run multiple times, computing different data each time -- later runs depending on earlier runs. I don't need to a persisent "disk" form. Though I end up close to that, esp. with this proposed change, which eliminates some pointers. I already do this in parse.c. Parse.c being written in C++ and having the preprocessor, it was much easier to do. See how I #include m3-def.h multiple times, with different macros. Ugly perhaps but highly convenient and productive and easy to use.. I'm so tempted to write "m3toc" this way, but I'm trying to use the product. It is painful.. Anyway, so I'm using Modula-3, and I have this really tedious code to convert the m3cg calls into an in-memory format. That I'll then loop over multiple times. Due to the tedium, I build up the in-memory format in one shared module, that I'm calling M3CG_MultiPass. It is closely analogous to M3CG_Ops.i3: TYPE declare_exception_t = REF RECORD name: Name; arg_typeid: TypeUID; raise_proc: BOOLEAN; base: Var; offset: INTEGER END; PROCEDURE set_runtime_proc(self: T; name: Name; proc: Proc) = BEGIN self.Add(NEW(set_runtime_proc_t, name := name, proc := proc)); END set_runtime_proc; etc. so, now, for this in-memory format to be used by arbitrary other backends..I think these "Var" and "Proc" will become a sticking point. I think we need a backend-independent way to refer to them. I think INTEGER is the way. Like how M3CG_BinWr deals with them. The downside/cost is that in M3x86.m3, rather than directly using the pointer (after narrow), it will instead index into its own private array. For efficiency, maybe some hint would come to the backend early as to how large to make the arrays? Or maybe that defeats some of the design -- the single pass implementations. (m3front is many-pass anyway, right? All this single-pass stuff is kind of only theoretical, right? The compiler working-set is not small and fixed size, but rather scales with module size, right?) Anyway, ok, either way. Backends can just at-least-double their internal arrays as they go. One might suggest some sort of "multiple playback" based on M3CG_BinWr/BinRd. I'm open to that. But it has some extra unnneeded overhead vs. a typical in-memory representation. I'm also open to changing BinWr/BinRd to use a larger representation that requires "no" decoding. (I've sketched this out myself, but decided it is not a priority. BinWr/BinRd really only exist for licensing reasons and would likely go away if m3cc went away. I think. They are kind of otherwise useful, at least during development.) Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 16:24:42 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:24:42 +0000 Subject: [M3devel] why this loop/multipass stuff Message-ID: why this loop/multipass stuff So people understand where I'm going here -- and Tony asked. I had already written this up.. 1) in the past working on m3cc/parse.c, I found that "declare"s weren't always "topologically sorted". That is, like, I might see: given TYPE T1 = INTEGER; TYPE T2 = RECORD = a:T1; END; I might have seen type T1 declared before T2, or not. I'm not sure if it was record/fields or other relationships. I just remember that building up a map of typeid to data, and looking up it when needed, didn't always work, with one pass. So what I did is that I was willing to "skip" declares that referred to typeids I hadn't seen yet, and then loop again, hoping for progress at each loop (fewer skips). 2) It looks like declare_local/temp don't all come in "at the start" of a procedure. I'm not sure if I'm generating C or C++, but if I'm generating C, it helps to have declare_local/temp right after begin_procedure/block. Multiple passes let me make that so, without changing m3front. (Upon further looking..it seems this is mostly done right, but not for "try" blocks; maybe reasonable to change m3front here.). Now, generating C++ is an option, and I'm ignoring all declares currently anyway, so these don't force the matter, but: 3) nested functions. If a function contains any nested functions, I want to put all of its locals in a local struct, and use the address of that struct as the "static link". Or at least the locals that are referenced by nested functions. As well, m3front contains options as to the order of outputing functions. It uses one order for m3back and a different order for m3cc. I wouldn't mind if M3C.m3 worked either way. There are possibly other benefits that could be derived from multiple passes. Nothing offhand though. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Aug 22 16:28:44 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 22 Aug 2012 10:28:44 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 22, 2012, at 10:24 AM, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Was this all in aid of building up proper type descriptors? > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. Preferably these only, or else you will cause all locals to be in memory instead of registers. > As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 22 16:41:09 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 07:41:09 -0700 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822144109.43DA71A2096@async.async.caltech.edu> Jay I've been following what you've been writing with one eye. But I'm not clear exactly what framework you're fitting into. Are you processing the output of m3front? There's always m3tk. Since it is claimed it was used to write a compiler it ought to be able to process MODULEs but I've only ever used it to process INTERFACEs. What's the story about that? Mika Tony Hosking writes: > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=iso-8859-1 > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > >> why this loop/multipass stuff >>=20 >>=20 >> So people understand where I'm going here -- and Tony asked. I had = >already written this up.. >>=20 >>=20 >>=20 >> 1) in the past working on m3cc/parse.c, I found that "declare"s >> weren't always "topologically sorted". >>=20 >>=20 >> That is, like, I might see: >>=20 >> given TYPE T1 =3D INTEGER; >> TYPE T2 =3D RECORD =3D a:T1; END; >>=20 >>=20 >> I might have seen type T1 declared before T2, or not. >> I'm not sure if it was record/fields or other relationships. >> I just remember that building up a map of typeid to data, and >> looking up it when needed, didn't always work, with one pass. >>=20 >>=20 >> So what I did is that I was willing to "skip" declares >> that referred to typeids I hadn't seen yet, and then loop >> again, hoping for progress at each loop (fewer skips). > >Was this all in aid of building up proper type descriptors? > >> 2) It looks like declare_local/temp don't all come in >> "at the start" of a procedure. I'm not sure if I'm generating >> C or C++, but if I'm generating C, it helps to have declare_local/temp >> right after begin_procedure/block. Multiple passes let me make that >> so, without changing m3front. >> (Upon further looking..it seems this is mostly done right, but >> not for "try" blocks; maybe reasonable to change m3front here.). >>=20 >>=20 >> Now, generating C++ is an option, and I'm ignoring all declares >> currently anyway, so these don't force the matter, but: >>=20 >>=20 >> 3) nested functions. >> If a function contains any nested functions, I want to put all of >> its locals in a local struct, and use the address of that struct >> as the "static link". Or at least the locals that are referenced >> by nested functions. > >Preferably these only, or else you will cause all locals to be in memory = >instead of registers. > >> As well, m3front contains options as to >> the order of outputing functions. It uses one order for m3back and >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either >> way. >>=20 >>=20 >> There are possibly other benefits that could be derived from multiple = >passes. >> Nothing offhand though. >>=20 >>=20 >> - Jay > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=iso-8859-1 > >charset=3Diso-8859-1">style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">
class=3D"Apple-style-span" color=3D"#0000FF">class=3D"Apple-style-span" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Antony = >Hoskingface=3D"Gill Sans">'Gill Sans'; ">'Gill Sans'; "> |class=3D"Apple-converted-space"> class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; ">class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = >">Associate Professorstyle=3D"font-family: 'Gill Sans'; ">style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = >University
face=3D"GillSans-Light">style=3D"font-family: GillSans-Light; ">305 N. University Street | West = >Lafayette | IN 47907 | USA
class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Mobileclass=3D"Apple-style-span" face=3D"GillSans-Light">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-converted-space"> +1 765 427 = >5484
face=3D"GillSans-Light">
class=3D"khtml-block-placeholder">
>

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline"> >
>
On Aug 22, 2012, at 10:24 AM, Jay K <href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = >wrote:

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">why this loop/multipass stuff


So people = >understand where I'm going here -- and Tony asked. I had already written = >this up..



1) in the past working on m3cc/parse.c, I found = >that "declare"s
weren't always "topologically = >sorted".


That is, like, I might see:

given TYPE T1 =3D = >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = >seen type T1 declared before T2, or not.
I'm not sure if it was = >record/fields or other relationships.
I just remember that building = >up a map of typeid to data, and
looking up it when needed, didn't = >always work, with one pass.


So what I did is that I was = >willing to "skip" declares
that referred to typeids I hadn't seen = >yet, and then loop
again, hoping for progress at each loop (fewer = >skips).

Was this all in = >aid of building up proper type descriptors?

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = >the start" of a procedure. I'm not sure if I'm generating
C or C++, = >but if I'm generating C, it helps to have declare_local/temp
right = >after begin_procedure/block. Multiple passes let me make that
so, = >without changing m3front.
(Upon further looking..it seems this is = >mostly done right, but
not for "try" blocks; maybe reasonable to = >change m3front here.).


Now, generating C++ is an option, and = >I'm ignoring all declares
currently anyway, so these don't force the = >matter, but:


3) nested functions.
If a function contains = >any nested functions, I want to put all of
its locals in a local = >struct, and use the address of that struct
as the "static link". Or = >at least the locals that are referenced
by nested = >functions.

Preferably these = >only, or else you will cause all locals to be in memory instead of = >registers.

style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = >">
As well, m3front contains options as to
the order = >of outputing functions. It uses one order for m3back and
a different = >order for m3cc. I wouldn't mind if M3C.m3 worked = >either
way.


There are possibly other benefits that could = >be derived from multiple passes.
Nothing offhand = >though.


 - = >Jay

= > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- From jay.krell at cornell.edu Wed Aug 22 16:56:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:56:31 +0000 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: <20120822144109.43DA71A2096@async.async.caltech.edu> References: , <20120822144109.43DA71A2096@async.async.caltech.edu> Message-ID: I'm implementing M3CG_Ops.m3. I don't trust m3tk -- it isn't the compiler. I'm somewhat familiar with m3cg. Given another M3CG, I know how to plug in to cm3, config files, etc. (e.g. add a new target, AMD64_NT, that will use this. :) ) On the other hand...I have a little more insight this morning. Variables have an "uplevel" parameter -- are they accessed from nested functions. I should be able to link variables to their containing procedure, and therefore its level. So "load(var)" I guess can be: text := "" for i := current_level to var.proc.level do (* typically an empty loop *) text := text & "_static_link->" end; if uplevel AND current_level = var.proc.level (* relatively rare *) text := text & "_locals."; end; text := text & var.name; or somesuch. I think I should get back to M3C with the assumption I don't need multiple passes...fil in more pieces..see where I get... - Jay > To: jay.krell at cornell.edu > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] why this loop/multipass stuff > Date: Wed, 22 Aug 2012 07:41:09 -0700 > From: mika at async.caltech.edu > > Jay I've been following what you've been writing with one eye. > > But I'm not clear exactly what framework you're fitting into. Are you > processing the output of m3front? > > There's always m3tk. Since it is claimed it was used to write a compiler > it ought to be able to process MODULEs but I've only ever used it to > process INTERFACEs. What's the story about that? > > Mika > > Tony Hosking writes: > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/plain; > > charset=iso-8859-1 > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > > > >> why this loop/multipass stuff > >>=20 > >>=20 > >> So people understand where I'm going here -- and Tony asked. I had = > >already written this up.. > >>=20 > >>=20 > >>=20 > >> 1) in the past working on m3cc/parse.c, I found that "declare"s > >> weren't always "topologically sorted". > >>=20 > >>=20 > >> That is, like, I might see: > >>=20 > >> given TYPE T1 =3D INTEGER; > >> TYPE T2 =3D RECORD =3D a:T1; END; > >>=20 > >>=20 > >> I might have seen type T1 declared before T2, or not. > >> I'm not sure if it was record/fields or other relationships. > >> I just remember that building up a map of typeid to data, and > >> looking up it when needed, didn't always work, with one pass. > >>=20 > >>=20 > >> So what I did is that I was willing to "skip" declares > >> that referred to typeids I hadn't seen yet, and then loop > >> again, hoping for progress at each loop (fewer skips). > > > >Was this all in aid of building up proper type descriptors? > > > >> 2) It looks like declare_local/temp don't all come in > >> "at the start" of a procedure. I'm not sure if I'm generating > >> C or C++, but if I'm generating C, it helps to have declare_local/temp > >> right after begin_procedure/block. Multiple passes let me make that > >> so, without changing m3front. > >> (Upon further looking..it seems this is mostly done right, but > >> not for "try" blocks; maybe reasonable to change m3front here.). > >>=20 > >>=20 > >> Now, generating C++ is an option, and I'm ignoring all declares > >> currently anyway, so these don't force the matter, but: > >>=20 > >>=20 > >> 3) nested functions. > >> If a function contains any nested functions, I want to put all of > >> its locals in a local struct, and use the address of that struct > >> as the "static link". Or at least the locals that are referenced > >> by nested functions. > > > >Preferably these only, or else you will cause all locals to be in memory = > >instead of registers. > > > >> As well, m3front contains options as to > >> the order of outputing functions. It uses one order for m3back and > >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either > >> way. > >>=20 > >>=20 > >> There are possibly other benefits that could be derived from multiple = > >passes. > >> Nothing offhand though. > >>=20 > >>=20 > >> - Jay > > > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/html; > > charset=iso-8859-1 > > > > >charset=3Diso-8859-1"> >style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; ">
> > >color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-border-horizontal-spacing: 0px; = > >-webkit-border-vertical-spacing: 0px; = > >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; font-size: medium; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = > >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = > >normal; font-variant: normal; font-weight: normal; letter-spacing: = > >normal; line-height: normal; orphans: 2; text-indent: 0px; = > >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; ">
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: = > >after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; ">
>class=3D"Apple-style-span" color=3D"#0000FF"> >class=3D"Apple-style-span" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Antony = > >Hosking >face=3D"Gill Sans"> >'Gill Sans'; "> >'Gill Sans'; "> | >class=3D"Apple-converted-space">  >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; "> >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = > >">Associate Professor >style=3D"font-family: 'Gill Sans'; "> >style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = > >University
> face=3D"GillSans-Light"> >style=3D"font-family: GillSans-Light; ">305 N. University Street | West = > >Lafayette | IN 47907 | USA
>class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Mobile >class=3D"Apple-style-span" face=3D"GillSans-Light"> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-converted-space"> +1 765 427 = > >5484
>face=3D"GillSans-Light">
>class=3D"khtml-block-placeholder">
>>

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline"> > >
> >
On Aug 22, 2012, at 10:24 AM, Jay K < >href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = > >wrote:

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">why this loop/multipass stuff


So people = > >understand where I'm going here -- and Tony asked. I had already written = > >this up..



1) in the past working on m3cc/parse.c, I found = > >that "declare"s
weren't always "topologically = > >sorted".


That is, like, I might see:

given TYPE T1 =3D = > >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = > >seen type T1 declared before T2, or not.
I'm not sure if it was = > >record/fields or other relationships.
I just remember that building = > >up a map of typeid to data, and
looking up it when needed, didn't = > >always work, with one pass.


So what I did is that I was = > >willing to "skip" declares
that referred to typeids I hadn't seen = > >yet, and then loop
again, hoping for progress at each loop (fewer = > >skips).

Was this all in = > >aid of building up proper type descriptors?

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = > >the start" of a procedure. I'm not sure if I'm generating
C or C++, = > >but if I'm generating C, it helps to have declare_local/temp
right = > >after begin_procedure/block. Multiple passes let me make that
so, = > >without changing m3front.
(Upon further looking..it seems this is = > >mostly done right, but
not for "try" blocks; maybe reasonable to = > >change m3front here.).


Now, generating C++ is an option, and = > >I'm ignoring all declares
currently anyway, so these don't force the = > >matter, but:


3) nested functions.
If a function contains = > >any nested functions, I want to put all of
its locals in a local = > >struct, and use the address of that struct
as the "static link". Or = > >at least the locals that are referenced
by nested = > >functions.

Preferably these = > >only, or else you will cause all locals to be in memory instead of = > >registers.

>style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = > >">
As well, m3front contains options as to
the order = > >of outputing functions. It uses one order for m3back and
a different = > >order for m3cc. I wouldn't mind if M3C.m3 worked = > >either
way.


There are possibly other benefits that could = > >be derived from multiple passes.
Nothing offhand = > >though.


 - = > >Jay

= > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 17:57:24 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 16:57:24 +0100 (BST) Subject: [M3devel] Fw: SW Bug #1080 In-Reply-To: <20120822092850.cef2e188.wagner@elego.de> Message-ID: <1345651044.13430.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: I'm on the process of booting a cm3-5.8.6-REL but in the second phase bootstrap process (NOPTHREAD) I have broke the compiler with this message (before I have fixed 2 quake thread scripts: m3-libs/m3core/src/thread/m3makefile:13 drop not clause m3-libs/m3core/src/thread.quake:87 same as above and add at line 90: else ??? PosixKernelThreads = FALSE ??? KernelThreads = FALSE ) : root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts# ./do-pkg.sh buildship m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 /home/test/code/cm3-5.8.6-REL/scripts/pkgmap.sh -c "/usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' " m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 === package /home/test/code/cm3-5.8.6-REL/m3-libs/m3core === ?+++ /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? +++ *** *** runtime error: ***??? Segmentation violation - possible attempt to dereference NIL ***??? pc = 0x8295954 = NoteStackLocations + 0x34 in ../src/runtime/common/RTCollector.m3 *** Aborted ?*** execution of /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? failed *** root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts#??? Antony, you have been busy but can you give us a clue on what to fix if so in m3core/src/runtime/common/RTCollector.m3 Thanks in advance --- El mi?, 22/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] Fw: SW Bug #1080 Para: m3devel at elegosoft.com CC: rob.zietlow at tdstelecom.com Fecha: mi?rcoles, 22 de agosto, 2012 02:28 I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL.? I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship???when I attempt to run cvsup right now it hangs.? When I connect to the cvsupd it also hangs.???I have strace access from the PIDS if needed.? One of the threads I came across commented about building with userthreads.? I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system.? I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 19:37:12 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 18:37:12 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822110048.903811A207D@async.async.caltech.edu> Message-ID: <1345657032.70947.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: in fact why not 'M3 as target language' is not the chicken and egg problem here, who needs C,? LLVM, stuff like that (we eat our own stuff and that's all), why not M3 in AST and that's all, like JSON la Olivetti. Olivetti tried to do that to compile and describe each node in the AST as an Object, we should explore the idea. http://lostechies.com/evanhoff/2008/01/08/language-roots/ Besides look for the 7th paragraph (I admire every day DEC-SRC and Modula-3 ever more) Thanks in advance --- El mi?, 22/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 06:00 In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? ? ? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; >??? charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > >? a.. CLR:=20 > >? ? a.. Pro: industrial support=20 >? ? b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 >? ? c.. Con: Only Windows platform is really prime-time quality >? b.. LLVM: > >? ? a.. Pro: enthusiastic user community with charismatic leader=20 >? ? b.. Pro: serious backing from Apple=20 >? ? c.. Pro: many interesting performance improvements=20 >? ? d.. Con: somewhat complex interface=20 >? ? e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface >? c.. C-- > >? ? a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 >? ? b.. Pro: design is reasonably mature and reasonably clean=20 >? ? c.. Pro: supports accurate garbage collection=20 >? ? d.. Pro: most users report it is very easy to use=20 >? ? e.. Con: very small development team=20 >? ? f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 >? ? g.. Con: does not ship with a garbage collector=20 >? ? h.. Con: project has no future >? d.. C as target language > >? ? a.. Pro: looks easy=20 >? ? b.. Con: nearly impossible to get decent performance=20 >? ? c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Wed Aug 22 20:11:06 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:11:06 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822181106.GA14178@topoi.pooq.com> On Wed, Aug 22, 2012 at 02:24:42PM +0000, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Does that mean you have a table of typeid's and their meanings in RAM during your multiple passes? If so, you might as well read them all in at once, and then link them together in RAM afterward. Or are you doing something sneakier? Or are you so short on RAM that thie is impractical? And for recorcds that are used by having pointers to them, you can just emit 'struct foo;' near the beginning of the generated C file. The topological sorting can be restricted to types that actually contain others, rather then pointing to them. This would reduce the amount of topological sorting you need to do. In fact, if you have all your types in RAM, you can do the topological sort *while* you are writing the types out. Just look through a type before emitting it and recurively emit any component types first. Mark each type when you emit it so you don't emit it twice. > > > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). How are you intending to make all these available for garbage collection? You might be interested in looking at Gambit/C, a Scheme compiler that compiles Schene to C and seems to do a fairly good job of it. > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. I wanted to do this in my Algol 68 compiler when I was still considering using the LLVM API, but placing temporaries and the like into the struct too so as to make it easier to form a struct descriptor for it for the garbage collector to ues. But that meant I'd be adding fields to it while I was gernerating the code that used it, and LLVM objected to it. Ecen though its whole API is oriented to putting pieces of the parse tree in piecemeal, jumping all arounf the thing if necessary, it still demanded that types be completely defined before you could use them for anything much. In particular, they had to be completely defined before you could use the API to generate parse tree that used the field names of the structure. This pretty well made the LLVM API significantly less attractive for this project. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay > From hendrik at topoi.pooq.com Wed Aug 22 20:38:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:38:52 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822183852.GB14178@topoi.pooq.com> On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > a.. CLR: > > a.. Pro: industrial support > b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader > b.. Pro: serious backing from Apple > c.. Pro: many interesting performance improvements > d.. Con: somewhat complex interface > e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead. Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to. Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again. If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish. This interfered with building run-time data structures whose full contents would not be known until compilation was complete. > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > b.. Pro: design is reasonably mature and reasonably clean > c.. Pro: supports accurate garbage collection > d.. Pro: most users report it is very easy to use > e.. Con: very small development team > f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > g.. Con: does not ship with a garbage collector > h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API. Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward. Of course this could be done with LLVM too. About the garbage collector. The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution. The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. > d.. C as target language > > a.. Pro: looks easy > b.. Con: nearly impossible to get decent performance > c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay From dabenavidesd at yahoo.es Thu Aug 23 02:51:46 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 01:51:46 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822183852.GB14178@topoi.pooq.com> Message-ID: <1345683106.74499.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I was reading an article and one a good consequence of Using a type safe like CLR, C-- would be be able to detect deadlocks with type inference. http://arxiv.org/pdarf/1002.0942.pdf So it can be good to do that for safety reasons, but at the cost of type inference time, which I think you wouldn't like too much for JIT compilers. Thanks in advance --- El mi?, 22/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] higher level m3cg? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 13:38 On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > >???a.. CLR: > >? ???a.. Pro: industrial support >? ???b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter >? ???c.. Con: Only Windows platform is really prime-time quality >???b.. LLVM: > >? ???a.. Pro: enthusiastic user community with charismatic leader >? ???b.. Pro: serious backing from Apple >? ???c.. Pro: many interesting performance improvements >? ???d.. Con: somewhat complex interface >? ???e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead.? Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to.? Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again.? If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish.? This interfered with building run-time data structures whose full contents would not be known until compilation was complete. ? >???c.. C-- > >? ???a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code >? ???b.. Pro: design is reasonably mature and reasonably clean >? ???c.. Pro: supports accurate garbage collection >? ???d.. Pro: most users report it is very easy to use >? ???e.. Con: very small development team >? ???f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >? ???g.. Con: does not ship with a garbage collector >? ???h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API.? Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward.? Of course this could be done with LLVM too. About the garbage collector.? The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution.? The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. >???d.. C as target language > >? ???a.. Pro: looks easy >? ???b.. Con: nearly impossible to get decent performance >? ???c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > >? > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. >???It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: >???enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >???typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > >? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 09:24:05 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 09:24:05 +0200 Subject: [M3devel] I know, I know... Message-ID: I know this will probably be very dense subject, but? What about: TSPacketHeader = <* ENDIAN = BIG*>RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) tErrInd, (* Transport Error Indicator *) pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) transPrio: BITS 1 FOR [0..1]; pid: BITS 13 FOR PID; transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) cc: BITS 4 FOR Nibble; END; Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: TSPacketHeader = RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) pidHi: BITS 5 FOR [16_00..16_1f]; transPrio: BITS 1 FOR [0..1]; pusi, tErrInd: BITS 1 FOR BOOLEAN; pidLo: BITS 8 FOR [16_00..16_ff]; cc: BITS 4 FOR Nibble; afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) END; And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. Thanks in advance.. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hosking at cs.purdue.edu Thu Aug 23 15:30:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 23 Aug 2012 09:30:57 -0400 Subject: [M3devel] I know, I know... In-Reply-To: References: Message-ID: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> PACKED? Sent from my iPad On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > I know this will probably be very dense subject, but? What about: > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > tErrInd, (* Transport Error Indicator *) > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > transPrio: BITS 1 FOR [0..1]; > pid: BITS 13 FOR PID; > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > cc: BITS 4 FOR Nibble; > END; > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > TSPacketHeader = RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > pidHi: BITS 5 FOR [16_00..16_1f]; > transPrio: BITS 1 FOR [0..1]; > pusi, > tErrInd: BITS 1 FOR BOOLEAN; > > pidLo: BITS 8 FOR [16_00..16_ff]; > > cc: BITS 4 FOR Nibble; > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > END; > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > Thanks in advance.. > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > From wagner at elego.de Thu Aug 23 16:24:03 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:24:03 +0200 Subject: [M3devel] CVSup hangs, was: Re: SW Bug #1080 In-Reply-To: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> References: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> Message-ID: <20120823162403.a2b0f83b.wagner@elego.de> On Wed, 22 Aug 2012 09:57:04 +0000 "Zietlow, Rob" wrote: > I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo > > I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. I think it might be helpful if you run the program in a debugger (gdb, m3gdb) and get a stack backtrace when it hangs, then post it to the list. Another approach might be to check if compilation with current sources will yield better results. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elego.de Thu Aug 23 16:33:19 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:33:19 +0200 Subject: [M3devel] recent and old bug reports Message-ID: <20120823163319.35c5bd77.wagner@elego.de> Is anybody out there with some time who could look into these issues: #1275 RTCollector problem when NOPTHREAD RT used libs 5.8.6 support wagner new 08/22/12 #1080 CVSUPD server hangs if used with -C option sys 5.8-RC3 sw-bug wagner new 12/29/09 #1148 CVSup stalls on several platforms misc 5.8.6 CM3 Release 5.9 sw-bug wagner new 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Thu Aug 23 17:25:28 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 17:25:28 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <3E9D81B8-0FC4-404A-A439-BEB21627F30E@m3w.org> packing, order, alignment. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 3:30 PM, Tony Hosking wrote: > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > >> I know this will probably be very dense subject, but? What about: >> >> TSPacketHeader = <* ENDIAN = BIG*>RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> tErrInd, (* Transport Error Indicator *) >> pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> transPrio: BITS 1 FOR [0..1]; >> pid: BITS 13 FOR PID; >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> cc: BITS 4 FOR Nibble; >> END; >> >> Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> >> Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> >> TSPacketHeader = RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> >> pidHi: BITS 5 FOR [16_00..16_1f]; >> transPrio: BITS 1 FOR [0..1]; >> pusi, >> tErrInd: BITS 1 FOR BOOLEAN; >> >> pidLo: BITS 8 FOR [16_00..16_ff]; >> >> cc: BITS 4 FOR Nibble; >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> END; >> >> And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> >> Thanks in advance.. >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 19:07:47 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:07:47 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741667.48136.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 23 19:08:03 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:08:03 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741683.90835.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:29:00 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:29:00 +0200 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> You mean, for system programming I have to use C? -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:26 PM, Jay K wrote: > How about you just use "bytes" or integers and helper functions like this: > > > typedef struct _BIT_FIELD { > UINT8 ByteOffset; > UINT8 BitOffset; > UINT8 BitCount; > } BIT_FIELD; > > > HRESULT > ExtractBitField(__in const BIT_FIELD* BitField, > __in const UINT8* Bytes, > __in size_t Size, > __out_opt UINT8* Value) > /*++ > Extract a bitfield from an array of bytes, as specified > by a byte offset, a bit offset within the first byte, and a bit count. > The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. > The bitfield can definitely cross byte boundaries. > --*/ > { > UINT8 const bitCount = BitField->BitCount; > UINT8 const bitOffset = BitField->BitOffset; > UINT8 const byteOffset = BitField->ByteOffset; > UINT8 const FF = (UINT8)~(UINT8)0; > BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); > UINT8 const secondBitOffset = (8 - bitOffset); > UINT8 localValue = { 0 }; > HRESULT error = { 0 }; > if (Value) > *Value = 0; > if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) > { > error = E_INVALIDARG; > goto Exit; > } > if ((byteOffset + twoBytes) >= Size) > { > error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); > goto Exit; > } > localValue = (Bytes[byteOffset] >> bitOffset); > if (twoBytes) > { > localValue &= ~(FF << secondBitOffset); > localValue |= (Bytes[byteOffset + 1] << secondBitOffset); > } > localValue &= ~(FF << bitCount); > error = 0; > if (Value) > *Value = localValue; > Exit: > return error; > } > > > arrays of bytes are really the only way to control the layout. > Look at how GNU binutils works, for example... > Bitfields in C don't yield predictable/portable layout either. > > > - Jay > > > From: hosking at cs.purdue.edu > > Date: Thu, 23 Aug 2012 09:30:57 -0400 > > To: dragisha at m3w.org > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] I know, I know... > > > > PACKED? > > > > Sent from my iPad > > > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > > > I know this will probably be very dense subject, but. What about: > > > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > tErrInd, (* Transport Error Indicator *) > > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > > transPrio: BITS 1 FOR [0..1]; > > > pid: BITS 13 FOR PID; > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > cc: BITS 4 FOR Nibble; > > > END; > > > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > > > TSPacketHeader = RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > > transPrio: BITS 1 FOR [0..1]; > > > pusi, > > > tErrInd: BITS 1 FOR BOOLEAN; > > > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > > > cc: BITS 4 FOR Nibble; > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > END; > > > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > > > Thanks in advance.. > > > -- > > > Divided by a common language > > > > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Thu Aug 23 21:26:58 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 19:26:58 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: How about you just use "bytes" or integers and helper functions like this: typedef struct _BIT_FIELD { UINT8 ByteOffset; UINT8 BitOffset; UINT8 BitCount; } BIT_FIELD; HRESULT ExtractBitField(__in const BIT_FIELD* BitField, __in const UINT8* Bytes, __in size_t Size, __out_opt UINT8* Value) /*++ Extract a bitfield from an array of bytes, as specified by a byte offset, a bit offset within the first byte, and a bit count. The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. The bitfield can definitely cross byte boundaries. --*/ { UINT8 const bitCount = BitField->BitCount; UINT8 const bitOffset = BitField->BitOffset; UINT8 const byteOffset = BitField->ByteOffset; UINT8 const FF = (UINT8)~(UINT8)0; BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); UINT8 const secondBitOffset = (8 - bitOffset); UINT8 localValue = { 0 }; HRESULT error = { 0 }; if (Value) *Value = 0; if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) { error = E_INVALIDARG; goto Exit; } if ((byteOffset + twoBytes) >= Size) { error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); goto Exit; } localValue = (Bytes[byteOffset] >> bitOffset); if (twoBytes) { localValue &= ~(FF << secondBitOffset); localValue |= (Bytes[byteOffset + 1] << secondBitOffset); } localValue &= ~(FF << bitCount); error = 0; if (Value) *Value = localValue; Exit: return error; } arrays of bytes are really the only way to control the layout.Look at how GNU binutils works, for example...Bitfields in C don't yield predictable/portable layout either. - Jay > From: hosking at cs.purdue.edu > Date: Thu, 23 Aug 2012 09:30:57 -0400 > To: dragisha at m3w.org > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > I know this will probably be very dense subject, but. What about: > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > tErrInd, (* Transport Error Indicator *) > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > transPrio: BITS 1 FOR [0..1]; > > pid: BITS 13 FOR PID; > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > cc: BITS 4 FOR Nibble; > > END; > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > TSPacketHeader = RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > transPrio: BITS 1 FOR [0..1]; > > pusi, > > tErrInd: BITS 1 FOR BOOLEAN; > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > cc: BITS 4 FOR Nibble; > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > END; > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > Thanks in advance.. > > -- > > Divided by a common language > > > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:42:51 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:42:51 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> Message-ID: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > You mean, for system programming I have to use C? > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:26 PM, Jay K wrote: > >> How about you just use "bytes" or integers and helper functions like this: >> >> >> typedef struct _BIT_FIELD { >> UINT8 ByteOffset; >> UINT8 BitOffset; >> UINT8 BitCount; >> } BIT_FIELD; >> >> >> HRESULT >> ExtractBitField(__in const BIT_FIELD* BitField, >> __in const UINT8* Bytes, >> __in size_t Size, >> __out_opt UINT8* Value) >> /*++ >> Extract a bitfield from an array of bytes, as specified >> by a byte offset, a bit offset within the first byte, and a bit count. >> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >> The bitfield can definitely cross byte boundaries. >> --*/ >> { >> UINT8 const bitCount = BitField->BitCount; >> UINT8 const bitOffset = BitField->BitOffset; >> UINT8 const byteOffset = BitField->ByteOffset; >> UINT8 const FF = (UINT8)~(UINT8)0; >> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >> UINT8 const secondBitOffset = (8 - bitOffset); >> UINT8 localValue = { 0 }; >> HRESULT error = { 0 }; >> if (Value) >> *Value = 0; >> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >> { >> error = E_INVALIDARG; >> goto Exit; >> } >> if ((byteOffset + twoBytes) >= Size) >> { >> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >> goto Exit; >> } >> localValue = (Bytes[byteOffset] >> bitOffset); >> if (twoBytes) >> { >> localValue &= ~(FF << secondBitOffset); >> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >> } >> localValue &= ~(FF << bitCount); >> error = 0; >> if (Value) >> *Value = localValue; >> Exit: >> return error; >> } >> >> >> arrays of bytes are really the only way to control the layout. >> Look at how GNU binutils works, for example... >> Bitfields in C don't yield predictable/portable layout either. >> >> >> - Jay >> >> > From: hosking at cs.purdue.edu >> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >> > To: dragisha at m3w.org >> > CC: m3devel at elegosoft.com >> > Subject: Re: [M3devel] I know, I know... >> > >> > PACKED? >> > >> > Sent from my iPad >> > >> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >> > >> > > I know this will probably be very dense subject, but. What about: >> > > >> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > tErrInd, (* Transport Error Indicator *) >> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> > > transPrio: BITS 1 FOR [0..1]; >> > > pid: BITS 13 FOR PID; >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > cc: BITS 4 FOR Nibble; >> > > END; >> > > >> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> > > >> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> > > >> > > TSPacketHeader = RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > >> > > pidHi: BITS 5 FOR [16_00..16_1f]; >> > > transPrio: BITS 1 FOR [0..1]; >> > > pusi, >> > > tErrInd: BITS 1 FOR BOOLEAN; >> > > >> > > pidLo: BITS 8 FOR [16_00..16_ff]; >> > > >> > > cc: BITS 4 FOR Nibble; >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > END; >> > > >> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> > > >> > > Thanks in advance.. >> > > -- >> > > Divided by a common language >> > > >> > > Dragi?a Duri? >> > > dragisha at m3w.org >> > > >> > > >> > > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Thu Aug 23 21:51:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 12:51:15 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: <20120823195115.80C121A2094@async.async.caltech.edu> Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" > > >--Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I have a friend, working for large software company and he just recently = >ported some message router from SPARC to Linux/Intel=85 Lots of "network = >order" data, same as with my current projects. Network, communication in = >general, network order is everywhere. > >Insisting on some god-given data ordering is a bit=85 What is nice word = >here? :) Outlandish? Outimeish? > >This is not first time I started discussion like this here. Every single = >time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >as I have shown in my example, with Modula-3 code. With a bit of effort = >I can make it almost-transparent (subfolder/platform) over various = >platforms. Of course, as I am developing a full product (not software to = >be run on arbitrary platform) I don't have to worry about too many = >platforms. One is enough here, but I still think Modula-3 can benefit, = >and a lot, if it supported explicit byte/bit ordering/packing/aligning = >pragmas. > >Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >we must write piece of software in C to be what? Compatible with API's = >all systems are compatible with without any hassle.=20 > >TIA >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From dragisha at m3w.org Thu Aug 23 21:59:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:59:46 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like? GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas of your own choice (or comments, or auxiliary data > from somewhere) and then generates code using Word.T that can process > byte streams and return the RECORD you declared, in portable Modula-3. > I doubt it's very difficult. m3tk is cool! > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" >> >> >> --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I have a friend, working for large software company and he just recently = >> ported some message router from SPARC to Linux/Intel=85 Lots of "network = >> order" data, same as with my current projects. Network, communication in = >> general, network order is everywhere. >> >> Insisting on some god-given data ordering is a bit=85 What is nice word = >> here? :) Outlandish? Outimeish? >> >> This is not first time I started discussion like this here. Every single = >> time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >> as I have shown in my example, with Modula-3 code. With a bit of effort = >> I can make it almost-transparent (subfolder/platform) over various = >> platforms. Of course, as I am developing a full product (not software to = >> be run on arbitrary platform) I don't have to worry about too many = >> platforms. One is enough here, but I still think Modula-3 can benefit, = >> and a lot, if it supported explicit byte/bit ordering/packing/aligning = >> pragmas. >> >> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >> we must write piece of software in C to be what? Compatible with API's = >> all systems are compatible with without any hassle.=20 >> >> TIA >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 22:34:09 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 21:34:09 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <2BADF386-2CFB-43EE-B809-F965AE351CD2@gmail.com> Message-ID: <1345754049.2762.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: just If you care for me, I don't have another machine to make bootstrap archives from newer compilers (that I don't have), and I haven't been following the trunck since when this stuff started to happen with CVSup (I think I still have the "mist stable RC3 or so"), so technically and morally, I have a dilemma, would need a lottery to make compilers work in so a long time when upgrading the compiler (maybe not). Manually I don't feel it's possible either as I lack the knowledge to do that either, so that's why I needed bootstrap archives but nobody answered here so... Thanks in advance --- El jue, 23/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] recent and old bug reports Para: "Daniel Alejandro Benavides D." CC: "m3devel at elegosoft.com" , "Olaf Wagner" Fecha: jueves, 23 de agosto, 2012 13:50 There are a number if bugs in the release that have since been fixed in the trunk. Best to upgrade from sources before trying to diagnose.? Sent from my iPhone On Aug 23, 2012, at 13:07, "Daniel Alejandro Benavides D." wrote: Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 00:12:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 22:12:48 +0000 Subject: [M3devel] I know, I know... In-Reply-To: References: , , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, Message-ID: I don't doubt that Modula-3 can handle it. And I don't doubt that C can handle it. But in both cases I find 1) the details beyond my attention span 2) definitely in the C case and very possibly in the Modula-3 case, not portable. You could learn the rules on one platform/compiler, only to produce code that doesn't work on others. Why bother learning them therefore? If you just want efficient storage in memory and don't care about the layout, then bitfields are ok. If you need to control the layout to interoperate with another system..even though bitfields are supposedly designed for this..I'm not sure I'd bother. C bitfields don't have portable layout. Do you suppose that Modula-3 ones do? Maybe there are in fact an obvious set of rules that all compilers follow? I don't know. Arrays of bytes have predictable layout. Look at how binutils reads/writes object files. It does like this: typedef struct elf_header_packed { char signature[4]; char architecture[4]; char offset_to_foo[4]; } elf_header_packed; typedef struct elf_header { long signature; long architecture; long offset_to_foo; } elf_header; and uses some pack/unpack functions to convert. They are solving a slightly different problem. They don't have bitfields necessarily to contend with, but just packing/size of integers that have a number of bits that are power of 2 (8, 16, 32, 64). - Jay From: dragisha at m3w.org Date: Thu, 23 Aug 2012 21:59:46 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] I know, I know... I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like. GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E Content-Type: multipart/alternative; boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1250 I have a friend, working for large software company and he just recently = ported some message router from SPARC to Linux/Intel=85 Lots of "network = order" data, same as with my current projects. Network, communication in = general, network order is everywhere. Insisting on some god-given data ordering is a bit=85 What is nice word = here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single = time Jay explains to me how Modula-3 cannot handle it. I am handling it, = as I have shown in my example, with Modula-3 code. With a bit of effort = I can make it almost-transparent (subfolder/platform) over various = platforms. Of course, as I am developing a full product (not software to = be run on arbitrary platform) I don't have to worry about too many = platforms. One is enough here, but I still think Modula-3 can benefit, = and a lot, if it supported explicit byte/bit ordering/packing/aligning = pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = we must write piece of software in C to be what? Compatible with API's = all systems are compatible with without any hassle.=20 TIA -- Divided by a common language Dragi=9Aa Duri=E6 dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Aug 24 01:07:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 16:07:44 -0700 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120823230744.509A61A2094@async.async.caltech.edu> Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before. If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline. Then I'd write a template file that allows me to specify said Scheme file in m3makefile. The advantage of this approach is that it doesn't rely on any compiler internals or special output files. And it's at least as powerful as any pragmas you might want to add. Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways). I only define two tables, you can imagine what the rest looks like. It's for a real-time financial trading application, if you were wondering. The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL. The code generator is only 1,197 lines of Scheme and that is including liberal documentation. (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.) The whole thing is added to the program by adding a single line to m3makefile. Mika (define aux-mm-ordr-ctrl (make-table "aux_mm_ordr_ctrl" 'client (make-field "symbol" 'varchar 'not-null) (make-index (list "symbol")) (make-field "type" 'varchar 'not-null) (make-field "expiry" 'varchar 'not-null) (make-field "exchange" 'varchar 'not-null) (make-field "currency" 'varchar 'not-null) (make-field "action" 'varchar 'not-null) (make-field "quantity" 'double-precision 'not-null) (make-field "order_type" 'varchar 'not-null) (make-field "benchmark_price" 'double-precision) (make-field "time_allowance" 'double-precision) (make-field "model_name" 'varchar) (make-index (list "model_name")) (make-field "id" 'integer) )) (define clean (make-table "clean" 'server (make-field "tabl" 'varchar 'not-null) (make-field "rowid" 'integer 'not-null) (make-field "client" 'varchar 'not-null) (make-index (list "client")) (make-index (list "tabl")) (make-index (list "tabl" "rowid")) (make-index (list "client" "tabl")))) (define gcoms (make-database "gcoms" clean code model instrument strategy model-data model-code ordr ordr_status output eror fill aux-instrument aux-ordr aux-ordr-params aux-market-segment aux-last-prices aux-mm-ordr-ctrl aux-mm-associations )) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future. And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult. m3tk is cool! >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- From jay.krell at cornell.edu Fri Aug 24 03:54:29 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 23 Aug 2012 18:54:29 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: Look at the code in m3core or libm3 that picks apart floats? The unsafe/loophole part isn't relevant. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 23, 2012, at 12:42 PM, Dragi?a Duri? wrote: > I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. > > Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? > > This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. > > Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. > > TIA > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > >> You mean, for system programming I have to use C? >> >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> >> On Aug 23, 2012, at 9:26 PM, Jay K wrote: >> >>> How about you just use "bytes" or integers and helper functions like this: >>> >>> >>> typedef struct _BIT_FIELD { >>> UINT8 ByteOffset; >>> UINT8 BitOffset; >>> UINT8 BitCount; >>> } BIT_FIELD; >>> >>> >>> HRESULT >>> ExtractBitField(__in const BIT_FIELD* BitField, >>> __in const UINT8* Bytes, >>> __in size_t Size, >>> __out_opt UINT8* Value) >>> /*++ >>> Extract a bitfield from an array of bytes, as specified >>> by a byte offset, a bit offset within the first byte, and a bit count. >>> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >>> The bitfield can definitely cross byte boundaries. >>> --*/ >>> { >>> UINT8 const bitCount = BitField->BitCount; >>> UINT8 const bitOffset = BitField->BitOffset; >>> UINT8 const byteOffset = BitField->ByteOffset; >>> UINT8 const FF = (UINT8)~(UINT8)0; >>> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >>> UINT8 const secondBitOffset = (8 - bitOffset); >>> UINT8 localValue = { 0 }; >>> HRESULT error = { 0 }; >>> if (Value) >>> *Value = 0; >>> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >>> { >>> error = E_INVALIDARG; >>> goto Exit; >>> } >>> if ((byteOffset + twoBytes) >= Size) >>> { >>> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >>> goto Exit; >>> } >>> localValue = (Bytes[byteOffset] >> bitOffset); >>> if (twoBytes) >>> { >>> localValue &= ~(FF << secondBitOffset); >>> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >>> } >>> localValue &= ~(FF << bitCount); >>> error = 0; >>> if (Value) >>> *Value = localValue; >>> Exit: >>> return error; >>> } >>> >>> >>> arrays of bytes are really the only way to control the layout. >>> Look at how GNU binutils works, for example... >>> Bitfields in C don't yield predictable/portable layout either. >>> >>> >>> - Jay >>> >>> > From: hosking at cs.purdue.edu >>> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >>> > To: dragisha at m3w.org >>> > CC: m3devel at elegosoft.com >>> > Subject: Re: [M3devel] I know, I know... >>> > >>> > PACKED? >>> > >>> > Sent from my iPad >>> > >>> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >>> > >>> > > I know this will probably be very dense subject, but. What about: >>> > > >>> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > tErrInd, (* Transport Error Indicator *) >>> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pid: BITS 13 FOR PID; >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > cc: BITS 4 FOR Nibble; >>> > > END; >>> > > >>> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >>> > > >>> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >>> > > >>> > > TSPacketHeader = RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > >>> > > pidHi: BITS 5 FOR [16_00..16_1f]; >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pusi, >>> > > tErrInd: BITS 1 FOR BOOLEAN; >>> > > >>> > > pidLo: BITS 8 FOR [16_00..16_ff]; >>> > > >>> > > cc: BITS 4 FOR Nibble; >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > END; >>> > > >>> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >>> > > >>> > > Thanks in advance.. >>> > > -- >>> > > Divided by a common language >>> > > >>> > > Dragi?a Duri? >>> > > dragisha at m3w.org >>> > > >>> > > >>> > > >>> > > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Aug 24 03:54:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 02:54:27 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120823230744.509A61A2094@async.async.caltech.edu> Message-ID: <1345773267.73869.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: No, I don't agree there isn't compiler internal intended to break interface that's other story but we don't fail compiled IRs at least that we know anything (at least the DEC-SRC front end needs to recompile every interface separately of any Module, so you can change compilers and still work the same clock speed should be equal in whatever configuration). We should take advantage of that inherent parallelism? I guess but we would need a good distributed quake machine. Alignment and endianness are same story about I don't mind that I just care for our actual clock speed of our machines instead of thinking in polymorphic type instructions etc. We eat our own stuff, VAX style machine type, everything else is just? head ache: http://mail-index.netbsd.org/current-users/1999/04/27/0032.html Thanks in advance --- El jue, 23/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] I know, I know... Para: "Dragi?a Duri?" CC: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 18:07 Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before.? If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline.? Then I'd write a template file that allows me to specify said Scheme file in m3makefile.? The advantage of this approach is that it doesn't rely on any compiler internals or special output files.? And it's at least as powerful as any pragmas you might want to add.? Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways).? I only define two tables, you can imagine what the rest looks like.? It's for a real-time financial trading application, if you were wondering.? The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL.? The code generator is only 1,197 lines of Scheme and that is including liberal documentation.? (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.)? The whole thing is added to the program by adding a single line to m3makefile. ? ???Mika (define aux-mm-ordr-ctrl ? (make-table "aux_mm_ordr_ctrl" 'client ? ? (make-field "symbol"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-index (list "symbol")) ? ? (make-field "type"? ? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "expiry"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "exchange"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "currency"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "action"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "quantity"? ? ? ? 'double-precision 'not-null) ? ? (make-field "order_type"? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "benchmark_price" 'double-precision) ? ? (make-field "time_allowance"? 'double-precision) ? ? (make-field "model_name"? ? ? 'varchar) ? ? (make-index (list "model_name")) ? ? (make-field "id"? ? ? ? ? ? ? 'integer) )) (define clean ? (make-table "clean" 'server ???(make-field "tabl"? ? ? ? ? 'varchar 'not-null) ???(make-field "rowid"? ? ? ???'integer 'not-null) ???(make-field "client"? ? ? ? 'varchar 'not-null) ? ? ???(make-index (list "client")) ???(make-index (list "tabl")) ???(make-index (list "tabl" "rowid")) ???(make-index (list "client" "tabl")))) ? ? (define gcoms ? (make-database "gcoms" ? ? ? ? ? ? ? ???clean ? ? ? ? ? ? ? ???code model instrument strategy model-data ? ? ? ? ? ? ? ???model-code ? ? ? ? ? ? ? ???ordr ordr_status ? ? ? ? ? ? ? ???output eror ? ? ? ? ? ? ? ???fill ? ? ? ? ? ? ? ???aux-instrument ? ? ? ? ? ? ? ???aux-ordr ? ? ? ? ? ? ? ???aux-ordr-params ? ? ? ? ? ? ? ???aux-market-segment ? ? ? ? ? ? ? ???aux-last-prices ? ? ? ? ? ? ? ???aux-mm-ordr-ctrl ? ? ? ? ? ? ? ???aux-mm-associations )) ? ? ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ??? (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; >??? boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; >??? charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future.? And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult.? m3tk is cool! >>=20 >>? ? Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> ??? boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> ??? charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; >??? charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; >??? filename=signature.asc >Content-Type: application/pgp-signature; >??? name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 24 16:00:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 24 Aug 2012 10:00:52 -0400 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120824140052.GA8309@topoi.pooq.com> On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice. If they are ignored on an implementation that doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone. Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away. It may be a deficiency we can live with. -- hendrik From dabenavidesd at yahoo.es Fri Aug 24 17:07:35 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 16:07:35 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> Message-ID: <1345820855.90388.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: In SPIN M3 you could VIEW any type as a bit string, but they also had a different compiler, so it's difficult to compare, in any case the thing written like that is UNSAFE in the data abstraction sense and so it's kind of easy see? that structure pattern in any other language with UNSAFE patterns. I like more structural typing in the sense if all are different you can't assign them, if you can then it's because they are not mismatched and obey the semantic rules of the language. Interesting combination of both would make a better RT though. Thanks in advance --- El vie, 24/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] I know, I know... Para: m3devel at elegosoft.com Fecha: viernes, 24 de agosto, 2012 09:00 On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future.? And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice.? If they are ignored on an implementation that? doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone.? Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away.? It may be a deficiency we can live with. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:43:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:43:44 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, <20120824140052.GA8309@topoi.pooq.com> Message-ID: Modula-3 has bit-mapping. Encoding it in functions that takes offsets and sizes is not hard. Using the language features gives you a choice of either - portable layout that doesn't likely match C - unportable layout that might match C At least unportable in terms of endianness -- again, for better and worse, see how we pick apart floats. It works, but the declarations are duplicated for little endian vs. big endian. Such knowledge does traditionally live in a compiler and its "core" libraries/runtime, but I'd still like to minimize it, maybe eliminate it. Neither is a ggeat choice I think. The compiler had tried to match target-specific layout/alignment rules. Whether it got them correct, I don't know. I relaxed the code because it was hard to know if it was right or wrong and just didn't seem worth maintaining -- i.e. porting to new systems. - Jay > Date: Fri, 24 Aug 2012 10:00:52 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > > (you can just use Word for that). > > > > But I think the point was that a compiler back-end might not necessarily > > obey your packing instructions? > > > > Here's what I think you should do... it would be super useful, both to > > you and other people in the future. And completely independent of > > compilers. > > > > Write a something-or-other that uses m3tk to read your RECORD type > > spec, plus pragmas > > Pragmas are definitely the wrong choice. If they are ignored on an > implementation that doesn't recognise them, this will be the wrong > semantics. > > Coding all the extraction and insertion functions is obscure, picky, and > error-prone. Bit-mapping is one of the reasons people use systems > languages, and the lack of it is a deficiency in Modula 3. > > Not that I'm advocating rewriting the compiler right away. It may be a > deficiency we can live with. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:48:02 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:48:02 +0000 Subject: [M3devel] Modula-3 bitfields Message-ID: Given: struct foo { union { struct { unsigned a : 16; unsigned b : 16; } s; unsigned c; } u; } foo = { {1, 2} }; printf("%X %X\n", c.u.s.a, c.u.s.b); What do people expect? Do people really have a mental model as to what the layout rules are for bitfields? Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? Furthermore, if presented with: struct foo { unsigned a : 16; unsigned b : 16; }; void F1(struct foo* f); printf("%X %X\n", f->a, f->b); Foo = RECORD a, b: BITS 16 FOR INTEGER END; <*EXTERNAL F1*> PROCEDURE F1(VAR foo: Foo); VAR f := Foo{1,2}; F1(f); What do people expect? That is, 1) Does anyone really internalize the layout rules for bitfields? 2) Does anyone really need them to line up with C? I ask because, it seems to me that in m3front/m3middle, if Little_endian were removed and assumed either always true or always false, everything would actually just work. The only thing that would go wrong is 1) interoperability with C..when there are bitfields 2) including in m3core where floats are picked apart 2b) But that's kind of good -- we could remove the little vs. big endian distinction there. See, I suspect but I am not certain the layout rules are actually well known to some small set of people, and they got encoded in m3front, in order for easier interoperability with C. Repeat the above exercises for more complicated examples, i.e. bitcounts that aren't multiples of 16. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri Aug 24 19:58:08 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 19:58:08 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: Message-ID: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> What mental model? We have network order. Everything network is bit streams, from left to right, most significant bits/bytes. No need for mental model here, any communication oriented ISO, ETSI, ITU standard? I don't understand why you insist on C syntax here. C has its own problems and reasons why it's at war with most things mental. Please refer to my initial example instead. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 7:48 PM, Jay K wrote: > printf("%X %X\n", c.u.s.a, c.u.s.b); > What do people expect? > Do people really have a mental model as to what the layout rules are for bitfields? > Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 20:10:35 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 11:10:35 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> Message-ID: <20120824181035.4EEFE1A207D@async.async.caltech.edu> The Green Book doesn't actually define alignment rules does it? Your example as I recall also specified precisely 32 bits. I can think of lots of ways things can go wrong. Why would you want to tie the hands of the compiler writer to a particular layout? Or clutter the interface with lots of pragmas... Yeah OK there was a day when that was the way to do things.... I think that time is past. The method I suggested could take advantage of situations where you know precisely how to construct the packed types. If you wanted. But it would also be entirely portable and independent of compilers. Heck you could generate C code at the same time and have free cross-language interoperability. (Or any other sets of languages.) You could also split bitfields in funny ways... Why would you do things any other way, really? Here's an example. A PCI Express DLLP is 32 bits of payload followed by a 16-bit CRC. Should I be able to write this as follows: DLLP = RECORD payload : BITS 32 FOR [0..16_ffffffff]; crc : BITS 16 FOR [0..16_ffff]; END; What then is the layout of ARRAY [0..1] OF DLLP; ? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Type: multipart/alternative; > boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >What mental model? > >We have network order. Everything network is bit streams, from left to = >right, most significant bits/bytes. No need for mental model here, any = >communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 > >I don't understand why you insist on C syntax here. C has its own = >problems and reasons why it's at war with most things mental. Please = >refer to my initial example instead. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 24, 2012, at 7:48 PM, Jay K wrote: > >> printf("%X %X\n", c.u.s.a, c.u.s.b); >> What do people expect? >> Do people really have a mental model as to what the layout rules are = >for bitfields? >> Are they obvious enough that every C compiler writer has actually = >implemented them the same and they become in a sense portable?=20 > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >mental model?

We have network order. Everything = >network is bit streams, from left to right, most significant bits/bytes. = >No need for mental model here, any communication oriented ISO, ETSI, ITU = >standard=E2=80=A6 

I don't understand why = >you insist on C syntax here. C has its own problems and reasons why it's = >at war with most things mental. Please refer to my initial example = >instead.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

class=3D"Apple-interchange-newline">
class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >expect?
Do people really have a mental model as to what the layout = >rules are for bitfields?
Are they obvious enough that every C = >compiler writer has actually implemented them the same and they become = >in a sense portable?class=3D"Apple-converted-space"> 
div>
= > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >=1GwZ >-----END PGP SIGNATURE----- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- From dragisha at m3w.org Fri Aug 24 21:20:03 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:20:03 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> As people in the world are divided in three groups, ones who can count and everybody else? So are we. I am in this group with this fancy idea of Modula-3 being system programming language. You don't agree? On Aug 24, 2012, at 8:10 PM, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Most probably. That's why I mentioned prgma in the first place. > > Your example as I recall also specified precisely 32 bits. > > I can think of lots of ways things can go wrong. > > Why would you want to tie the hands of the compiler writer to a particular > layout? Or clutter the interface with lots of pragmas? Pragma is good way to make things optional for some platforms, and to not change language definition. Platform specific, and working on AMD64 for example, is good enough. > > Yeah OK there was a day when that was the way to do things.... I think > that time is past. The method I suggested could take advantage of > situations where you know precisely how to construct the packed types. > If you wanted. But it would also be entirely portable and independent > of compilers. Heck you could generate C code at the same time and have > free cross-language interoperability. (Or any other sets of languages.) > > You could also split bitfields in funny ways... > > Why would you do things any other way, really? Being a Modula-2|3 programmer for 24 years obviously did funny things to me :). Like, I really started to like how high-level programs look like, esp ones doing various low-level stuff :). I like to think abstract, to write high-level, readable programs. Ones I don't have problems maintaining for prolonged time intervals. I don't need anything funny. I just need a way to represent, in a readable manner, standard communication data structures. So I can continue with my trade without C, > > Here's an example. A PCI Express DLLP is 32 bits of payload followed by > a 16-bit CRC. Should I be able to write this as follows: > > DLLP = RECORD > payload : BITS 32 FOR [0..16_ffffffff]; > crc : BITS 16 FOR [0..16_ffff]; > END; > > What then is the layout of > > ARRAY [0..1] OF DLLP; > Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? > ? > > Mika dd > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> What mental model? >> >> We have network order. Everything network is bit streams, from left to = >> right, most significant bits/bytes. No need for mental model here, any = >> communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 >> >> I don't understand why you insist on C syntax here. C has its own = >> problems and reasons why it's at war with most things mental. Please = >> refer to my initial example instead. >> >> -- >> Divided by a common language >> >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> >> On Aug 24, 2012, at 7:48 PM, Jay K wrote: >> >>> printf("%X %X\n", c.u.s.a, c.u.s.b); >>> What do people expect? >>> Do people really have a mental model as to what the layout rules are = >> for bitfields? >>> Are they obvious enough that every C compiler writer has actually = >> implemented them the same and they become in a sense portable?=20 >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >> mental model?

We have network order. Everything = >> network is bit streams, from left to right, most significant bits/bytes. = >> No need for mental model here, any communication oriented ISO, ETSI, ITU = >> standard=E2=80=A6 

I don't understand why = >> you insist on C syntax here. C has its own problems and reasons why it's = >> at war with most things mental. Please refer to my initial example = >> instead.

>> > color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">
--
Divided by a = >> common language

Dragi=C5=A1a = >> Duri=C4=87

= >>


> class=3D"Apple-interchange-newline"> >>
>>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

> class=3D"Apple-interchange-newline">
> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >> 16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >> expect?
Do people really have a mental model as to what the layout = >> rules are for bitfields?
Are they obvious enough that every C = >> compiler writer has actually implemented them the same and they become = >> in a sense portable?> class=3D"Apple-converted-space"> 
> div>
= >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Transfer-Encoding: 7bit >> Content-Disposition: attachment; >> filename=signature.asc >> Content-Type: application/pgp-signature; >> name=signature.asc >> Content-Description: Message signed with OpenPGP using GPGMail >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.18 (Darwin) >> >> iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >> fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >> tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >> 5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >> M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >> VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >> =1GwZ >> -----END PGP SIGNATURE----- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 21:28:59 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 12:28:59 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> Message-ID: <20120824192859.C3B801A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: ... >to me :). Like, I really started to like how high-level programs look = >like, esp ones doing various low-level stuff :). I like to think = >abstract, to write high-level, readable programs. Ones I don't have = >problems maintaining for prolonged time intervals. > > I don't need anything funny. I just need a way to represent, in a = >readable manner, standard communication data structures. So I can = >continue with my trade without C, > >>=20 >> Here's an example. A PCI Express DLLP is 32 bits of payload followed = >by >> a 16-bit CRC. Should I be able to write this as follows: >>=20 >> DLLP =3D RECORD >> payload : BITS 32 FOR [0..16_ffffffff]; >> crc : BITS 16 FOR [0..16_ffff]; >> END; >>=20 >> What then is the layout of=20 >>=20 >> ARRAY [0..1] OF DLLP; >>=20 > >Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika From dragisha at m3w.org Fri Aug 24 21:53:44 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:53:44 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824192859.C3B801A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> Message-ID: <14795BC4-4830-442C-A674-633197203234@m3w.org> What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: > > And what does that mean, exactly? No gaps allowed? What does it > mean, in particular, on a little-endian machine? > > What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > > What I am proposing isn't particularly difficult, it's not unportable, > either. And much more powerful than any sets of pragmas. Not tied to > your one architecture. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 05:06:06 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 03:06:06 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <14795BC4-4830-442C-A674-633197203234@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: > What mental model? > We have network order. Everything network is bit streams, from left to right, most > significant bits/bytes. Does any Modula-2 or Modula-3 or C compiler follow this model?For a little endian target? I'm skeptical.If it really is obvious, maybe we can make cm3 follow it?Even if it breaks bitfield interop with C?I would like to have m3front and M3C.m3 not have any target-endian dependentness.I do believe it would break pickles, and some floating point code in m3core/libm3. ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). In particular, standard C only allows bit fields to be of type "int" or "unsigned".However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? What does this look like: struct { unsigned a : 1; unsigned b : 32;};? - Jay From: dragisha at m3w.org Date: Fri, 24 Aug 2012 21:53:44 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Aug 25 07:35:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 22:35:51 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: <20120825053551.CB4BD1A207D@async.async.caltech.edu> Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. Mika Jay K writes: >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/plain; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > What mental model? > We have network order. Everything network is bit st= >reams=2C from left to right=2C most > significant bits/bytes. Does any Mod= >ula-2 or Modula-3 or C compiler follow this model?For a little endian targe= >t? I'm skeptical.If it really is obvious=2C maybe we can make cm3 follow it= >?Even if it breaks bitfield interop with C?I would like to have m3front and= > M3C.m3 not have any target-endian dependentness.I do believe it would brea= >k pickles=2C and some floating point code in m3core/libm3. ..because..you = >see.. I think there some mental model in C compiler writer's heads...that c= >m3 tries to adhere to=2C but which I might have diverged a bit from (look a= >t the history of Target.m3). In particular=2C standard C only allows bit f= >ields to be of type "int" or "unsigned".However it is a common extension to= > allow the type of a bitfield to also unsigned/signed char=2C short=2C long= >=2C and long long=2C and this does I believe affect the layout. Does that f= >it your mental model? In your model=2C must a 32bit bit field be aligned on= > a 32bit boundary? What does this look like: struct { unsigned a : 1=3B u= >nsigned b : 32=3B}=3B? - Jay > From: dragisha at m3w.org >Date: Fri=2C 24 Aug 2012 21:53:44 +0200 >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: Re: [M3devel] Modula-3 bitfields > >What you are proposing is to do a compiler-level job with a library. Just b= >ecause such a library exists does not mean it invalidates every other appro= >ach. And we are talking about one of most complicated pieces of Modula-3 sy= >stem=2C comparable in complexity with compiler. Of course=2C there are othe= >r approaches too=2C like some kind of interface language=2C to be processed= > into Modula-3 source code by cm3 build system. >Your latest example is good illustration for my proposal too. Another simpl= >e example to show a need for exact bit packing layout method. Thank you! >=0A= >--Divided by a common language >Dragi=B9a Duri=E6dragisha at m3w.org >=0A= >=0A= > >On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote: >And what does that mean=2C exactly? No gaps allowed? What does it >mean=2C in particular=2C on a little-endian machine? > >What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > >What I am proposing isn't particularly difficult=2C it's not unportable=2C >either. And much more powerful than any sets of pragmas. Not tied to >your one architecture. > > Mika > = > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/html; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > > >
 =3B>=3B What mental model= >?
 =3B>=3B We have network order. Everything network is bit s= >treams=2C from left to right=2C most
 =3B>=3B significant b= >its/bytes.
 =3B
 =3B
Does any Modula-= >2 or Modula-3 or C compiler follow this model?
For a little endia= >n target? I'm skeptical.
If it really is obvious=2C maybe we can = >make cm3 follow it?
Even if it breaks bitfield interop with C?iv>
I would like to have m3front and M3C.m3 not have any target-endian = >dependentness.
I do believe it would break pickles=2C and some fl= >oating point code in m3core/libm3.
 =3B ..because..you see.. = >I think there some mental model in C compiler writer's heads...that cm3 tri= >es to adhere to=2C but which I might have diverged a bit from (look at the = >history of Target.m3).
 =3B
 =3B
In p= >articular=2C standard C only allows bit fields to be of type "int" or "unsi= >gned".
However it is a common extension to allow the type of a bi= >tfield to also unsigned/signed char=2C short=2C long=2C and long long=2C an= >d this does I believe affect the layout. Does that fit your mental model? I= >n your model=2C must a 32bit bit field be aligned on a 32bit boundary?>
 =3B
What does this look like:
 =3B
= >
struct {
 =3B unsigned a : 1=3B
 =3B unsig= >ned b : 32=3B
}=3B
?
 =3B
 = >=3B
 =3B- Jay
 =3B
older">

From: dragisha at m3w.org
Date: Fri=2C= > 24 Aug 2012 21:53:44 +0200
To: mika at async.caltech.edu
CC: m3devel at el= >egosoft.com
Subject: Re: [M3devel] Modula-3 bitfields

What you ar= >e proposing is to do a compiler-level job with a library. Just because such= > a library exists does not mean it invalidates every other approach. And we= > are talking about one of most complicated pieces of Modula-3 system=2C com= >parable in complexity with compiler. Of course=2C there are other approache= >s too=2C like some kind of interface language=2C to be processed into Modul= >a-3 source code by cm3 build system.

Your latest ex= >ample is good illustration for my proposal too. Another simple example to s= >how a need for exact bit packing layout method. Thank you!

div>
=0A= >nt: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B white-space: norm= >al=3B border-collapse: separate=3B orphans: 2=3B widows: 2=3B" class=3D"ecx= >Apple-style-span"> none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B= > white-space: normal=3B border-collapse: separate=3B orphans: 2=3B widows: = >2=3B" class=3D"ecxApple-style-span">
>
--
Divided by a common language

Dr= >agi=B9a Duri=E6


newline">

=0A= >
=0A= >
On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote:
<= >br class=3D"ecxApple-interchange-newline">
ransform: none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacin= >g: 0px=3B white-space: normal=3B border-collapse: separate=3B orphans: 2=3B= > widows: 2=3B" class=3D"ecxApple-style-span">
And what does that mean=2C= > exactly?  =3BNo gaps allowed?  =3BWhat does it
mean=2C in parti= >cular=2C on a little-endian machine?

What about ARRAY [0..1] OF BITS= > 31 FOR [0..16_8fffffff] ?

What I am proposing isn't particularly di= >fficult=2C it's not unportable=2C
either.  =3BAnd much more powerful= > than any sets of pragmas.  =3BNot tied to
your one architecture.>
 =3B =3B =3B =3BMika

v>
>= > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_-- From dragisha at m3w.org Sat Aug 25 08:29:12 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:29:12 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: Did I said in my first post, pragma? With pragma you don't activate a behavior unless you need something pragma provides. Locally, in pragma's scope. So you will not break anything, anywhere outside pragma's scope. gm2 is very precise with bitpacking and aligning. I did not check for byte/bit order, or anything gm2 recently, too much work (most of it in Modula-3). As this is perfectly logical system programming requirement I do not doubt Gaius will be positive to such proposal. What good is bitfield interop when our pointers are 64bit aligned and GCC's are 32bit? And when every interop question is answered with "write critical piece in C" ? Proposed pragma would not break anything, it is local behavior. I believe in "mental model", but I also believe in evolution. Do you? :) All of us have decades of experience accumulated, and my decades changed my models, as slow as it is. I see this pattern, I see this need. I see solution through alternate bit/byte packing into structures inside pragma scope. As for your example, outside of proposed pragma scope it will be 32 bit aligned. But, if it is a 64 bit value - in GCC it will be 32bit aligned, while cm3 will (most probably) align it to 64bit. There goes interop. Ooops. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 5:06 AM, Jay K wrote: > > What mental model? > > We have network order. Everything network is bit streams, from left to right, most > > significant bits/bytes. > > > Does any Modula-2 or Modula-3 or C compiler follow this model? > For a little endian target? I'm skeptical. > If it really is obvious, maybe we can make cm3 follow it? > Even if it breaks bitfield interop with C? > I would like to have m3front and M3C.m3 not have any target-endian dependentness. > I do believe it would break pickles, and some floating point code in m3core/libm3. > ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). > > > In particular, standard C only allows bit fields to be of type "int" or "unsigned". > However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? > > What does this look like: > > struct { > unsigned a : 1; > unsigned b : 32; > }; > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sat Aug 25 08:31:16 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:31:16 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825053551.CB4BD1A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> Message-ID: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 09:22:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 07:22:44 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: 1) Most likely, whatever you want, you can get with Modula-3. You can at least attain the compactness you want. How you match up with some specified protocol, independently implementable e.g. with C and shifting/masking, less certain. 2) If this is for some network/persistant I/O, the result will not be portable. At least due to endianness. Possibly due to alignment. 3) I claim that interop with C remains a strong feature of Modula-3, but not necessarily regarding bitfields. There is some attempt at such interop, but I'd just as soon ignore it. I'd even give an error if types with bitfields are used with <* external *> functions. For my libc interop, I'm very conservative -- records contain just longint and integer, sorted by size -- there is never any padding, except maybe at the end. I looked a bit through "windows.i3" and "xlib.i3" and decided they seem pretty save too. 3) I am faced with generating target-independent C. This is the worry/doubt I inject/tangent into your line of questioning. You ask about one thing, I bring up something else (kind of like Daniel..) m3front checks endian, making it almost impossible. I fear there isn't a good answer here. I kind of would just like to a) remove the check b) adjust {libm3, m3core}/float accordingly. Alternatively, change m3front/m3cg some. Have the result say: #ifdef LITTLE_ENDIAN something #else something else #endif Perhaps the endian check can be removed if the target platform is "C"? Relevant aside: "By default", in C#, layout is undefined. It could sort the fields by size/name, I think, and be valid. Or put stuff near each other based on runtime profiling. Is Modula-3 this abstracted too?? But there are at least two ways to control the layout. You can assert that it must be "sequential". You can give actual offsets. I don't think C# even has bitfields though. - Jay Subject: Re: [M3devel] Modula-3 bitfields From: dragisha at m3w.org Date: Sat, 25 Aug 2012 08:31:16 +0200 CC: jay.krell at cornell.edu; m3devel at elegosoft.com To: mika at async.caltech.edu I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote:Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 13:22:17 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 13:22:17 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <21696995-DEE3-4DD7-AAB5-13C0EC62316B@m3w.org> On Aug 25, 2012, at 9:22 AM, Jay K wrote: > 1) Most likely, whatever you want, you can get with Modula-3. > You can at least attain the compactness you want. > How you match up with some specified protocol, independently > implementable e.g. with C and shifting/masking, less certain. Modula-3 can do everything needed, in ad-hoc and generally totally unportable manner :). > > > 2) If this is for some network/persistant I/O, the result will > not be portable. At least due to endianness. Possibly due to alignment. Not if we have means to force both endianness of data and alignment. > > > 3) I claim that interop with C remains a strong feature of Modula-3, > but not necessarily regarding bitfields. There is some attempt at such > interop, but I'd just as soon ignore it. I'd even give an error > if types with bitfields are used with <* external *> functions. I don't need much bitfield related interop with C. 32bit aligned pointers remain on my wish list :). > > > For my libc interop, I'm very conservative -- records contain just > longint and integer, sorted by size -- there is never any padding, > except maybe at the end. > I looked a bit through "windows.i3" and "xlib.i3" and decided > they seem pretty save too. libc interop is great, and I depend a lot on it. Thank you. > 3) I am faced with generating target-independent C. > This is the worry/doubt I inject/tangent into your line of questioning. > You ask about one thing, I bring up something else (kind of like Daniel..) :) This realy is a tangent. As my proposal was (localized to single structures) pragma, I don't see a link here. dd > m3front checks endian, making it almost impossible. > I fear there isn't a good answer here. > I kind of would just like to a) remove the check b) adjust > {libm3, m3core}/float accordingly. > Alternatively, change m3front/m3cg some. > Have the result say: > > > #ifdef LITTLE_ENDIAN > something > #else > something else > #endif > > > Perhaps the endian check can be removed if the target platform is "C"? > > > Relevant aside: > "By default", in C#, layout is undefined. > It could sort the fields by size/name, I think, and be valid. > Or put stuff near each other based on runtime profiling. > Is Modula-3 this abstracted too?? > But there are at least two ways to control the layout. > You can assert that it must be "sequential". > You can give actual offsets. > I don't think C# even has bitfields though. > > > - Jay > > > > > Subject: Re: [M3devel] Modula-3 bitfields > From: dragisha at m3w.org > Date: Sat, 25 Aug 2012 08:31:16 +0200 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: mika at async.caltech.edu > > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hendrik at topoi.pooq.com Sat Aug 25 21:05:48 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 25 Aug 2012 15:05:48 -0400 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <20120825190548.GA9036@topoi.pooq.com> On Sat, Aug 25, 2012 at 08:31:16AM +0200, Dragi?a Duri? wrote: > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to > C". Also - proposed pragma is local behavior. Not active unless you > need it, but when you need it you will be really grateful for it :). To adequately describe data structures defined elsewhere (as in communications standards) we need to be able to specify alignment, width, padding, and endianness. It doesn't need to be global for all the data in a program -- in fact, it had better not. It should be specified explicitly for each data structure or data item for which it matters. And it's a crucial part of the semantics of the program, not just something that the compiler should handle if it happens to recognise a pragma -- it needs to be part of the panguage proper. How the compiler chooses to store other data, which is not so draconically constrained, is entirely up to it. That said, it's still an open question whether to do something like this at all. But if it is to be done, it has to be done right. -- hendrik From microcode at zoho.com Sat Aug 25 21:13:17 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sat, 25 Aug 2012 19:13:17 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <20120825191317.GA3747@zoho.com> On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) From dragisha at m3w.org Sat Aug 25 22:08:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:08:46 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825191317.GA3747@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> Message-ID: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Systems Programming with Modula 3 Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote: > On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: >> The Green Book doesn't actually define alignment rules does it? > > Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison > but it's white and grimy (looks like it was a garage copy!) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 22:20:56 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 20:20:56 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: Microsoft C and gcc have a variety of ways to control this stuff. command line for the entire compilation #pragma pack/push/pop for a run of code __declspec(align) on individual types __unaligned on individual fields various __attribute__ probably on types and fields Which brings me back to ...array of bytes, very clear, very portable... If you need compactness but not exact layout, C and Modula-3 do fine. If you need a portable predictable layout, other than an array of bytes, they both largely fail. - Jay From: dragisha at m3w.org Date: Sat, 25 Aug 2012 22:08:46 +0200 To: microcode at zoho.com CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields Systems Programming with Modula 3Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote:On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 22:56:41 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:56:41 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> I am in no doubt about general inadequacies of C et al? What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 10:20 PM, Jay K wrote: > Which brings me back to ...array of bytes, very clear, very portable... > > If you need compactness but not exact layout, C and Modula-3 do fine. > If you need a portable predictable layout, other than an array of bytes, they both largely fail. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Aug 25 23:55:19 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 25 Aug 2012 14:55:19 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: <20120825215519.BA0431A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" > > >--Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I am in no doubt about general inadequacies of C et al=85 > >What I hate is an idea of setters and getters for my bitfields. With = >pragma, compiler generated load/store will behave as needed. Everything = >transparent and just-working. Until you have some data that is PDP-endian :-) Or want to split a field across a boundary that you're not supposed to be able to split a field across. Or have a field that's semantically scattered/gathered. Can your proposed scheme represent IEEE floating point? I agree it is a very useful idea, I agree it has to be done "right". But my definition of doing it "right" would be so demanding you'd be crazy to do it in the compiler.... Mika > >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From jay.krell at cornell.edu Sun Aug 26 04:59:26 2012 From: jay.krell at cornell.edu (Jay) Date: Sat, 25 Aug 2012 19:59:26 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: Other languages let you have "properties" that let you hide get/set functions behind field access syntax... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 25, 2012, at 1:56 PM, Dragi?a Duri? wrote: > I am in no doubt about general inadequacies of C et al? > > What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 10:20 PM, Jay K wrote: > >> Which brings me back to ...array of bytes, very clear, very portable... >> >> If you need compactness but not exact layout, C and Modula-3 do fine. >> If you need a portable predictable layout, other than an array of bytes, they both largely fail. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 26 06:56:59 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 26 Aug 2012 04:56:59 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <20120826045659.GA27610@zoho.com> On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- _ _ ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. From dragisha at m3w.org Sun Aug 26 09:03:14 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:03:14 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825215519.BA0431A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> <20120825215519.BA0431A207D@async.async.caltech.edu> Message-ID: <6AB35AF7-0D1A-40EA-B03A-E270B393CC8A@m3w.org> Right and simple are two faces of that coin. Every long walk has its first step. And so on :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 11:55 PM, Mika Nystrom wrote: > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" >> >> >> --Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I am in no doubt about general inadequacies of C et al=85 >> >> What I hate is an idea of setters and getters for my bitfields. With = >> pragma, compiler generated load/store will behave as needed. Everything = >> transparent and just-working. > > Until you have some data that is PDP-endian :-) > > Or want to split a field across a boundary that you're not supposed to > be able to split a field across. > > Or have a field that's semantically scattered/gathered. > > Can your proposed scheme represent IEEE floating point? > > I agree it is a very useful idea, I agree it has to be done "right". > But my definition of doing it "right" would be so demanding you'd be > crazy to do it in the compiler.... > > Mika > >> >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sun Aug 26 09:05:57 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:05:57 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <20120826045659.GA27610@zoho.com> Message-ID: <634D1148-0033-4037-A20E-AAAD05F0CFF1@m3w.org> Books don't disappear, usually :). They go out of print and move to afterlife in antiquity shops and forgotten shelves :). Google for table of contest of SPwM3 and match with names of SRC reports. I don't have a book with me right now so I can't write it out for you, sorry. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 26, 2012, at 6:56 AM, microcode at zoho.com wrote: > On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >> >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers) >> > > Oh thanks. I'll try to get a copy of that if it is good. I did download most > of the SRC stuff but have no idea when I'll have time to look at it. > > I'm trying to get a few books before they all disappear. > > -- > _ _ > ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ > | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> > |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Mon Aug 27 04:18:29 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 02:18:29 +0000 Subject: [M3devel] multiple evaluation hazard in backend? Message-ID: Tony, does m3front tend or tend not to ask the backend to evaluate the same expression multiple times, or does it tend to allocate a temporary if it would do so? It's just that my current approach, if not given temporaries, may tend to evaluate expressions multiple times that either must not (for correctness) or ought not (for efficiency). I guess I'll learn soon enough. My current "stack" is just TEXTs. Very simple. If I had to eliminate multiple evaluations I'd have to consider a few alternatives: 1) for every, push, do something like "{ T s0 = expr;" for every pop "}" That would require I always "use before pop", not "use not after pop" as I tend do. That isn't always easy. The above is also wrong. Instead of "s0", it'd be "sN" where N is the current stack depth, then "get(n)" would, like, get N-n. 2) Alternatively, if I knew the maximum stack depth such as by making two passes, I could have an array of unions. That would make casts less verbose -- they'd be via union member access, if no size change. 2b) I could maintain a stack pointer at compile time or runtime. Runtime easier but slower. I still might want to record types with my texts, like to avoid extra casts. Or maybe m3front does that already? Anyway, for now I can ignore it. I'm just starting in on function calls, and I still haven't handled nested functions. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 04:37:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 03:37:31 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> Message-ID: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email and I'll tell you url) in a public site, SPwM3 is almost but last chapter is presumably in other hands (the so-called lost video tapes), and the other two books are (OK are not totally on-line, but bigger part are available from archive.org). I was trying hard to compile SPwM3 in one file, but the more I see it, I guess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was already available, we have already the C.1 - Introduction, C. 2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An? Introduction to programming with Threads (available from MS-Research in C# syntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are almost in source tree, it will just miss C. 5 - Thread Synchronization (but is one of the smallest < 20 p. I believe we can make a revised copy based on current Interface specification after we get a copy of ESC/M3). It needs a bit of work I guess, but if some kind soul is wanting to do this I will let him do it. The only needed book is Algorithms in Modula-3 (but there is a German script of very good quality from U Aachen) so, it's not too bad but sure will be nice to recover it from somewhere (I heard it was used to develop complete Algorithm Animations, which would be perfect for Mentor) Thanks in advance --- El s?b, 25/8/12, microcode at zoho.com escribi?: De: microcode at zoho.com Asunto: Re: [M3devel] Modula-3 bitfields Para: m3devel at elegosoft.com Fecha: s?bado, 25 de agosto, 2012 23:56 On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- ? ? ? ? _? ? ? ? ? ? ? ? ? ? ? ? ? ???_? ? ? ._ _ _ <_> ___? _ _? ___? ___? ___? _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_|? \___/\_|_.\___/\___|\___. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 05:08:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 03:08:31 +0000 Subject: [M3devel] race conditions in lock-free code... Message-ID: We have race conditions, like this, m3core/src/Text.m3: VAR fromCharCache := ARRAY CHAR OF T {NIL, ..}; (* 1-char texts *) PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; fromCharCache[c] := Text8.New (buf); END; RETURN fromCharCache [c] END FromChar; It should be: PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; WITH a = Text8.New (buf) DO MemoryBarrier(); fromCharCache[c] := a; END; END; RETURN fromCharCache [c] END FromChar; to ensure that all of Text8.New() finishes before the assignment to fromCharCache[c] is made. Can the compiler somehow catch these? I fear they are a small epidemic. For a long time people didn't realize where all the compiler and processor could reorder. Do we have the right constructs by now to fix them? I think we do. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 06:19:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:19:15 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> References: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Message-ID: <20120827041915.569CD1A207D@async.async.caltech.edu> If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3. The Handbook of Programming Languages has a chapter on Modula-3 as well. Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- From mika at async.caltech.edu Mon Aug 27 06:23:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:23:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: Message-ID: <20120827042351.141B81A207D@async.async.caltech.edu> Yeah it's a race condition, in theory. But is it important? If you get the wrong copy, there will be two TEXTs representing one CHAR. But no one ever said FromChar was guaranteed to return the same pointer when you call it with the same CHAR... Mika Jay K writes: >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >We have race conditions=2C like this=2C m3core/src/Text.m3: > > >VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > > >PROCEDURE FromChar (c: CHAR): T =3D > VAR buf: ARRAY [0..0] OF CHAR=3B > BEGIN > IF fromCharCache [c] =3D NIL THEN > buf [0] :=3D c=3B > fromCharCache[c] :=3D Text8.New (buf)=3B > END=3B > RETURN fromCharCache [c] > END FromChar=3B > > >It should be: > > >PROCEDURE FromChar (c: CHAR): T =3D >=0A= > VAR buf: ARRAY [0..0] OF CHAR=3B >=0A= > BEGIN >=0A= > IF fromCharCache [c] =3D NIL THEN >=0A= > buf [0] :=3D c=3B >=0A= > WITH a =3D Text8.New (buf) DO > MemoryBarrier()=3B > fromCharCache[c] :=3D a=3B > END=3B >=0A= > END=3B >=0A= > RETURN fromCharCache [c] >=0A= > END FromChar=3B >=0A= > > >to ensure that all of Text8.New() finishes before the assignment to fromCha= >rCache[c] is made. > > >Can the compiler somehow catch these? >I fear they are a small epidemic. >For a long time people didn't realize where all the compiler and processor = >could reorder. > > >Do we have the right constructs by now to fix them? >I think we do. > > > - Jay > = > >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
We have race conditions=2C like = >this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >r (c: CHAR): T =3D
=0A= > =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > =3B BEGIN
=0A= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > =3B =3B =3B END=3B
=0A= > =3B =3B =3B RETURN fromCharCache [c]
=0A= > =3B END FromChar=3B
=0A= >

to ensure that all of Text8.New() finishes before the assignment to= > fromCharCache[c] is made.


Can the compiler somehow catch these?= >
I fear they are a small epidemic.
For a long time people didn't real= >ize where all the compiler and processor could reorder.


Do we ha= >ve the right constructs by now to fix them?
I think we do.


-= > Jay
>= > >--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 06:44:21 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 04:44:21 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 07:32:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 22:32:48 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827042351.141B81A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> Message-ID: <20120827053248.A1B3F1A207D@async.async.caltech.edu> Sorry, I can't seem to quote your message. You say that Text8.New's result can be assigned to fromCharCache[c] before Text8.New is done running. How is that possible? Surely Text8.New has to calculate its result before it can be assigned anywhere. Yes possibly FromChar allocates space for the result of Text8.New but the assignment of whatever appears in that space can't happen until *after* Text8.New has run. Unless you are saying that the running of Text8.New and FromChar are intertwined somehow..? I know compilers do some odd things but that doesn't sound right... If Text8.New took fromCharCache[c] as a VAR parameter you might be right, though. But it doesn't... Am I missing something? Mika Mika Nystrom writes: > >Yeah it's a race condition, in theory. But is it important? If you get >the wrong copy, there will be two TEXTs representing one CHAR. But no >one ever said FromChar was guaranteed to return the same pointer when >you call it with the same CHAR... > > Mika > >Jay K writes: >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >>We have race conditions=2C like this=2C m3core/src/Text.m3: >> >> >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >> VAR buf: ARRAY [0..0] OF CHAR=3B >> BEGIN >> IF fromCharCache [c] =3D NIL THEN >> buf [0] :=3D c=3B >> fromCharCache[c] :=3D Text8.New (buf)=3B >> END=3B >> RETURN fromCharCache [c] >> END FromChar=3B >> >> >>It should be: >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >>=0A= >> VAR buf: ARRAY [0..0] OF CHAR=3B >>=0A= >> BEGIN >>=0A= >> IF fromCharCache [c] =3D NIL THEN >>=0A= >> buf [0] :=3D c=3B >>=0A= >> WITH a =3D Text8.New (buf) DO >> MemoryBarrier()=3B >> fromCharCache[c] :=3D a=3B >> END=3B >>=0A= >> END=3B >>=0A= >> RETURN fromCharCache [c] >>=0A= >> END FromChar=3B >>=0A= >> >> >>to ensure that all of Text8.New() finishes before the assignment to fromCha= >>rCache[c] is made. >> >> >>Can the compiler somehow catch these? >>I fear they are a small epidemic. >>For a long time people didn't realize where all the compiler and processor = >>could reorder. >> >> >>Do we have the right constructs by now to fix them? >>I think we do. >> >> >> - Jay >> = >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
We have race conditions=2C like = >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >>r (c: CHAR): T =3D
=0A= >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= >> =3B BEGIN
=0A= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= >> =3B =3B =3B END=3B
=0A= >> =3B =3B =3B RETURN fromCharCache [c]
=0A= >> =3B END FromChar=3B
=0A= >>

to ensure that all of Text8.New() finishes before the assignment to= >> fromCharCache[c] is made.


Can the compiler somehow catch these?= >>
I fear they are a small epidemic.
For a long time people didn't real= >>ize where all the compiler and processor could reorder.


Do we ha= >>ve the right constructs by now to fix them?
I think we do.


-= >> Jay
>>= >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 08:27:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 06:27:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827053248.A1B3F1A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: The processor can reorder. Across call/ret at least in general. The compiler can also reorder, if you have whole-program-optimization, link-time-code-gen (Microsoft "LTCG"), link-time-optimization (gcc "LTO"). (We don't, but real world systems certainly do.) - Jay > To: jay.krell at cornell.edu; m3devel at elegosoft.com; mika at async.caltech.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Sun, 26 Aug 2012 22:32:48 -0700 > From: mika at async.caltech.edu > > Sorry, I can't seem to quote your message. > > You say that Text8.New's result can be assigned to fromCharCache[c] > before Text8.New is done running. How is that possible? Surely > Text8.New has to calculate its result before it can be assigned > anywhere. Yes possibly FromChar allocates space for the result of > Text8.New but the assignment of whatever appears in that space can't > happen until *after* Text8.New has run. Unless you are saying that > the running of Text8.New and FromChar are intertwined somehow..? > I know compilers do some odd things but that doesn't sound right... > > If Text8.New took fromCharCache[c] as a VAR parameter you might be > right, though. But it doesn't... > > Am I missing something? > > Mika > > Mika Nystrom writes: > > > >Yeah it's a race condition, in theory. But is it important? If you get > >the wrong copy, there will be two TEXTs representing one CHAR. But no > >one ever said FromChar was guaranteed to return the same pointer when > >you call it with the same CHAR... > > > > Mika > > > >Jay K writes: > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/plain; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >>We have race conditions=2C like this=2C m3core/src/Text.m3: > >> > >> > >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >> BEGIN > >> IF fromCharCache [c] =3D NIL THEN > >> buf [0] :=3D c=3B > >> fromCharCache[c] :=3D Text8.New (buf)=3B > >> END=3B > >> RETURN fromCharCache [c] > >> END FromChar=3B > >> > >> > >>It should be: > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >>=0A= > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >>=0A= > >> BEGIN > >>=0A= > >> IF fromCharCache [c] =3D NIL THEN > >>=0A= > >> buf [0] :=3D c=3B > >>=0A= > >> WITH a =3D Text8.New (buf) DO > >> MemoryBarrier()=3B > >> fromCharCache[c] :=3D a=3B > >> END=3B > >>=0A= > >> END=3B > >>=0A= > >> RETURN fromCharCache [c] > >>=0A= > >> END FromChar=3B > >>=0A= > >> > >> > >>to ensure that all of Text8.New() finishes before the assignment to fromCha= > >>rCache[c] is made. > >> > >> > >>Can the compiler somehow catch these? > >>I fear they are a small epidemic. > >>For a long time people didn't realize where all the compiler and processor = > >>could reorder. > >> > >> > >>Do we have the right constructs by now to fix them? > >>I think we do. > >> > >> > >> - Jay > >> = > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/html; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
We have race conditions=2C like = > >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= > >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = > >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= > >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= > >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= > >>r (c: CHAR): T =3D
=0A= > >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > >> =3B BEGIN
=0A= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= > >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= > >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= > >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B RETURN fromCharCache [c]
=0A= > >> =3B END FromChar=3B
=0A= > >>

to ensure that all of Text8.New() finishes before the assignment to= > >> fromCharCache[c] is made.


Can the compiler somehow catch these?= > >>
I fear they are a small epidemic.
For a long time people didn't real= > >>ize where all the compiler and processor could reorder.


Do we ha= > >>ve the right constructs by now to fix them?
I think we do.


-= > >> Jay
> >>= > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 09:17:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 00:17:44 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: <20120827071744.781621A207D@async.async.caltech.edu> Oh I see now. It's not that the processor reorders in any tricky way. It is that the write performed by Text8.New could be sitting around processor 1's cache indefinitely without actually getting written back to main memory. Processor 2 might see the new pointer, but when reading the memory pointed to, would get uninitialized memory. I think this can happen on Alpha but maybe not X86 (without further reorderings anyhow)? Any chance any of the pthreads bugs could be of this nature? How are the pthreads doing? I gathered some bugs were fixed but I haven't tried the system for a while. Mika Jay K writes: >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >The processor can reorder. Across call/ret at least in general. > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >. (We don't=2C but real world systems certainly do.) > > > - Jay > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >> From: mika at async.caltech.edu >>=20 >> Sorry=2C I can't seem to quote your message. >>=20 >> You say that Text8.New's result can be assigned to fromCharCache[c] >> before Text8.New is done running. How is that possible? Surely >> Text8.New has to calculate its result before it can be assigned >> anywhere. Yes possibly FromChar allocates space for the result of >> Text8.New but the assignment of whatever appears in that space can't >> happen until *after* Text8.New has run. Unless you are saying that >> the running of Text8.New and FromChar are intertwined somehow..? >> I know compilers do some odd things but that doesn't sound right... >>=20 >> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> right=2C though. But it doesn't... >>=20 >> Am I missing something? >>=20 >> Mika >>=20 >> Mika Nystrom writes: >> > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= >et >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no >> >one ever said FromChar was guaranteed to return the same pointer when >> >you call it with the same CHAR... >> > >> > Mika >> > >> >Jay K writes: >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >> >> >> >> >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >exts *) >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >> BEGIN >> >> IF fromCharCache [c] =3D3D NIL THEN >> >> buf [0] :=3D3D c=3D3B >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >> >> END=3D3B >> >> RETURN fromCharCache [c] >> >> END FromChar=3D3B >> >> >> >> >> >>It should be: >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >>=3D0A=3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >>=3D0A=3D >> >> BEGIN >> >>=3D0A=3D >> >> IF fromCharCache [c] =3D3D NIL THEN >> >>=3D0A=3D >> >> buf [0] :=3D3D c=3D3B >> >>=3D0A=3D >> >> WITH a =3D3D Text8.New (buf) DO >> >> MemoryBarrier()=3D3B >> >> fromCharCache[c] :=3D3D a=3D3B >> >> END=3D3B >> >>=3D0A=3D >> >> END=3D3B >> >>=3D0A=3D >> >> RETURN fromCharCache [c] >> >>=3D0A=3D >> >> END FromChar=3D3B >> >>=3D0A=3D >> >> >> >> >> >>to ensure that all of Text8.New() finishes before the assignment to fro= >mCha=3D >> >>rCache[c] is made. >> >> >> >> >> >>Can the compiler somehow catch these? >> >>I fear they are a small epidemic. >> >>For a long time people didn't realize where all the compiler and proces= >sor =3D >> >>could reorder. >> >> >> >> >> >>Do we have the right constructs by now to fix them? >> >>I think we do. >> >> >> >> >> >> - Jay >> >> =3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >> >> >> >>
We have race conditions= >=3D2C like =3D >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > CHAR OF=3D >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >(c: CHAR=3D >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >3B BEGIN
=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >nbsp=3D3B =3D >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > =3D3B =3D3B&n=3D >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >p=3D3B =3D3B=3D >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >che [c]
&nb=3D >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > FromCha=3D >> >>r (c: CHAR): T =3D3D
=3D0A=3D >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >> >> =3D3B BEGIN
=3D0A=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >B
=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >ew (buf) DO
&=3D >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >emoryBarrier()=3D3B<=3D >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >3B fromCharCache[c=3D >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>=3D0A=3D >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >> >> =3D3B END FromChar=3D3B
=3D0A=3D >> >>

to ensure that all of Text8.New() finishes before the assignmen= >t to=3D >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= >ese?=3D >> >>
I fear they are a small epidemic.
For a long time people didn't = >real=3D >> >>ize where all the compiler and processor could reorder.


Do w= >e ha=3D >> >>ve the right constructs by now to fix them?
I think we do.

r> -=3D >> >> Jay
>> >>=3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > = > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
The processor can reorder. Acros= >s call/ret at least in general.


The compiler can also reorder=2C= > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >stems certainly do.)


 =3B- Jay


yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >is done running. How is that possible? Surely
>=3B Text8.New has to = >calculate its result before it can be assigned
>=3B anywhere. Yes pos= >sibly FromChar allocates space for the result of
>=3B Text8.New but th= >e assignment of whatever appears in that space can't
>=3B happen until= > *after* Text8.New has run. Unless you are saying that
>=3B the runni= >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >mpilers do some odd things but that doesn't sound right...
>=3B
&g= >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >one ever said FromChar was guaranteed to return the same pointer when
&g= >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >=3B >=3B>=3BFor a long time people didn't realize where all the compile= >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >-311cd8c58979_--
>= > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 10:55:09 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:55:09 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 10:58:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:58:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu>, Message-ID: Also, a compiler is free to reorder this stuff as well. If it can see the body of the function and inline it. Without an explicit lock/fence/barrier of some kind, including a call to a function that the compiler cannot see the body of, the compiler can reorder. Our compiler doesn't see far, so that helps. But that is an implementation detail of our compiler. The C backend will make "WPO" / "LTCG" / "LTO" sort of more viable..and the compiler will then see more. (all names for the same thing -- whole program optimization, link time code generation, link time optimization) - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] race conditions in lock-free code... Date: Mon, 27 Aug 2012 08:55:09 +0000 Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 14:20:56 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 13:20:56 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120827041915.569CD1A207D@async.async.caltech.edu> Message-ID: <1346070056.46764.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yet, put another "A Theory of Objects" is almost begin to end in the web, however it misses just the justice with "Baby Modula-3" as it was used to check the type system of Modula-3, I believe but this research reports never came out (or the library didn't published it) so it's difficult to know what else is missing. In fact they wrote an accompanying software to it, a new implementation of Obliq based on first chapters of the book, don't know where it ended. Thanks --- El dom, 26/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Modula-3 bitfields Para: "Daniel Alejandro Benavides D." CC: m3devel at elegosoft.com Fecha: domingo, 26 de agosto, 2012 23:19 If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3.? The Handbook of Programming Languages has a chapter on Modula-3 as well. ? ???Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Aug 27 15:27:38 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 09:27:38 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. Sent from my iPad On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The processor can reorder. Across call/ret at least in general. >> >> >> The compiler can also reorder=2C if you have whole-program-optimization=2C = >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >> . (We don't=2C but real world systems certainly do.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >> edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >>> From: mika at async.caltech.edu >>> =20 >>> Sorry=2C I can't seem to quote your message. >>> =20 >>> You say that Text8.New's result can be assigned to fromCharCache[c] >>> before Text8.New is done running. How is that possible? Surely >>> Text8.New has to calculate its result before it can be assigned >>> anywhere. Yes possibly FromChar allocates space for the result of >>> Text8.New but the assignment of whatever appears in that space can't >>> happen until *after* Text8.New has run. Unless you are saying that >>> the running of Text8.New and FromChar are intertwined somehow..? >>> I know compilers do some odd things but that doesn't sound right... >>> =20 >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>> right=2C though. But it doesn't... >>> =20 >>> Am I missing something? >>> =20 >>> Mika >>> =20 >>> Mika Nystrom writes: >>>> >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= >> et >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no >>>> one ever said FromChar was guaranteed to return the same pointer when >>>> you call it with the same CHAR... >>>> >>>> Mika >>>> >>>> Jay K writes: >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>>> >>>>> >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >> exts *) >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> BEGIN >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> buf [0] :=3D3D c=3D3B >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >>>>> END=3D3B >>>>> RETURN fromCharCache [c] >>>>> END FromChar=3D3B >>>>> >>>>> >>>>> It should be: >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> =3D0A=3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> =3D0A=3D >>>>> BEGIN >>>>> =3D0A=3D >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> =3D0A=3D >>>>> buf [0] :=3D3D c=3D3B >>>>> =3D0A=3D >>>>> WITH a =3D3D Text8.New (buf) DO >>>>> MemoryBarrier()=3D3B >>>>> fromCharCache[c] :=3D3D a=3D3B >>>>> END=3D3B >>>>> =3D0A=3D >>>>> END=3D3B >>>>> =3D0A=3D >>>>> RETURN fromCharCache [c] >>>>> =3D0A=3D >>>>> END FromChar=3D3B >>>>> =3D0A=3D >>>>> >>>>> >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= >> mCha=3D >>>>> rCache[c] is made. >>>>> >>>>> >>>>> Can the compiler somehow catch these? >>>>> I fear they are a small epidemic. >>>>> For a long time people didn't realize where all the compiler and proces= >> sor =3D >>>>> could reorder. >>>>> >>>>> >>>>> Do we have the right constructs by now to fix them? >>>>> I think we do. >>>>> >>>>> >>>>> - Jay >>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> >>>>> >>>>> >>>>>
We have race conditions= >> =3D2C like =3D >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= >> CHAR OF=3D >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >> (c: CHAR=3D >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >> 3B BEGIN
=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >> nbsp=3D3B =3D >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= >>  =3D3B =3D3B&n=3D >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >> p=3D3B =3D3B=3D >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >> che [c]
&nb=3D >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= >> FromCha=3D >>>>> r (c: CHAR): T =3D3D
=3D0A=3D >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >>>>>  =3D3B BEGIN
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >> B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >> ew (buf) DO
&=3D >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >> emoryBarrier()=3D3B<=3D >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >> 3B fromCharCache[c=3D >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D >>>>>

to ensure that all of Text8.New() finishes before the assignmen= >> t to=3D >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= >> ese?=3D >>>>>
I fear they are a small epidemic.
For a long time people didn't = >> real=3D >>>>> ize where all the compiler and processor could reorder.


Do w= >> e ha=3D >>>>> ve the right constructs by now to fix them?
I think we do.

> r> -=3D >>>>> Jay
>>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
The processor can reorder. Acros= >> s call/ret at least in general.


The compiler can also reorder=2C= >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >> stems certainly do.)


 =3B- Jay


> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >> is done running. How is that possible? Surely
>=3B Text8.New has to = >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= >> e assignment of whatever appears in that space can't
>=3B happen until= >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >> one ever said FromChar was guaranteed to return the same pointer when
&g= >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >> -311cd8c58979_--
>> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 20:07:10 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 18:07:10 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: It is NOT benign. The global can be written before the initialization of what it points to is done. The result is wrong. It IS benign if multiple threads go and allocate separate multiple initialized TEXTs. That's not the problem. - Jay > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > From: hosking at cs.purdue.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Mon, 27 Aug 2012 09:27:38 -0400 > To: mika at async.caltech.edu > > So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. > > Sent from my iPad > > On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > > > Oh I see now. > > > > It's not that the processor reorders in any tricky way. It is that the > > write performed by Text8.New could be sitting around processor 1's cache > > indefinitely without actually getting written back to main memory. > > Processor 2 might see the new pointer, but when reading the memory pointed > > to, would get uninitialized memory. I think this can happen on Alpha but > > maybe not X86 (without further reorderings anyhow)? > > > > Any chance any of the pthreads bugs could be of this nature? How are the > > pthreads doing? I gathered some bugs were fixed but I haven't tried the > > system for a while. > > > > Mika > > > > > > Jay K writes: > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> The processor can reorder. Across call/ret at least in general. > >> > >> > >> The compiler can also reorder=2C if you have whole-program-optimization=2C = > >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >> . (We don't=2C but real world systems certainly do.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >> edu > >>> Subject: Re: [M3devel] race conditions in lock-free code... > >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >>> From: mika at async.caltech.edu > >>> =20 > >>> Sorry=2C I can't seem to quote your message. > >>> =20 > >>> You say that Text8.New's result can be assigned to fromCharCache[c] > >>> before Text8.New is done running. How is that possible? Surely > >>> Text8.New has to calculate its result before it can be assigned > >>> anywhere. Yes possibly FromChar allocates space for the result of > >>> Text8.New but the assignment of whatever appears in that space can't > >>> happen until *after* Text8.New has run. Unless you are saying that > >>> the running of Text8.New and FromChar are intertwined somehow..? > >>> I know compilers do some odd things but that doesn't sound right... > >>> =20 > >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >>> right=2C though. But it doesn't... > >>> =20 > >>> Am I missing something? > >>> =20 > >>> Mika > >>> =20 > >>> Mika Nystrom writes: > >>>> > >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= > >> et > >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >>>> one ever said FromChar was guaranteed to return the same pointer when > >>>> you call it with the same CHAR... > >>>> > >>>> Mika > >>>> > >>>> Jay K writes: > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >>>>> > >>>>> > >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >> exts *) > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> BEGIN > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> buf [0] :=3D3D c=3D3B > >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >>>>> END=3D3B > >>>>> RETURN fromCharCache [c] > >>>>> END FromChar=3D3B > >>>>> > >>>>> > >>>>> It should be: > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> =3D0A=3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> =3D0A=3D > >>>>> BEGIN > >>>>> =3D0A=3D > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> =3D0A=3D > >>>>> buf [0] :=3D3D c=3D3B > >>>>> =3D0A=3D > >>>>> WITH a =3D3D Text8.New (buf) DO > >>>>> MemoryBarrier()=3D3B > >>>>> fromCharCache[c] :=3D3D a=3D3B > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> RETURN fromCharCache [c] > >>>>> =3D0A=3D > >>>>> END FromChar=3D3B > >>>>> =3D0A=3D > >>>>> > >>>>> > >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= > >> mCha=3D > >>>>> rCache[c] is made. > >>>>> > >>>>> > >>>>> Can the compiler somehow catch these? > >>>>> I fear they are a small epidemic. > >>>>> For a long time people didn't realize where all the compiler and proces= > >> sor =3D > >>>>> could reorder. > >>>>> > >>>>> > >>>>> Do we have the right constructs by now to fix them? > >>>>> I think we do. > >>>>> > >>>>> > >>>>> - Jay > >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>
We have race conditions= > >> =3D2C like =3D > >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > >> CHAR OF=3D > >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >> (c: CHAR=3D > >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >> 3B BEGIN
=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >> nbsp=3D3B =3D > >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > >>  =3D3B =3D3B&n=3D > >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >> p=3D3B =3D3B=3D > >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >> che [c]
&nb=3D > >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > >> FromCha=3D > >>>>> r (c: CHAR): T =3D3D
=3D0A=3D > >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >>>>>  =3D3B BEGIN
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >> B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >> ew (buf) DO
&=3D > >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >> emoryBarrier()=3D3B<=3D > >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >> 3B fromCharCache[c=3D > >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D > >>>>>

to ensure that all of Text8.New() finishes before the assignmen= > >> t to=3D > >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= > >> ese?=3D > >>>>>
I fear they are a small epidemic.
For a long time people didn't = > >> real=3D > >>>>> ize where all the compiler and processor could reorder.


Do w= > >> e ha=3D > >>>>> ve the right constructs by now to fix them?
I think we do.

>> r> -=3D > >>>>> Jay
> >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- > >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
The processor can reorder. Acros= > >> s call/ret at least in general.


The compiler can also reorder=2C= > >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >> stems certainly do.)


 =3B- Jay


>> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >> is done running. How is that possible? Surely
>=3B Text8.New has to = > >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >> e assignment of whatever appears in that space can't
>=3B happen until= > >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >> one ever said FromChar was guaranteed to return the same pointer when
&g= > >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= > >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >> -311cd8c58979_--
> >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 20:34:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 11:34:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: <20120827183451.976971A2094@async.async.caltech.edu> yeah I agree. processor 1 performs... x := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent. If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive. In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2. (Or Exclusive, if P1 writes it back and invalidates.) This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global. The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice. I'd suspect that a multiprocessor 21264 could be. But I do agree it's a bug relative to the Alpha Architecture Manual. A very nasty one at that. Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily. The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way. It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory. I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature? How are t= >he >> > pthreads doing? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> > Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running. How is that possible? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere. Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run. Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though. But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>> Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory. But is it important? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>> Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>> END=3D3D3B >> >>>>> RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> WITH a =3D3D3D Text8.New (buf) DO >> >>>>> MemoryBarrier()=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running. How is that possible? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere. = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run. Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >t=3D3B IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >B RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B =3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B =3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily. The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way. It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere. Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory. But is it important? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= > buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = > <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B =3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From dabenavidesd at yahoo.es Mon Aug 27 23:54:16 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 22:54:16 +0100 (BST) Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <1346104456.75314.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I believe there shouldn't be so many for not being counted with fingers of one hand. The memory model was specified and model checked with TLA by Lamport and publicly available for bug finders contest. However being an UNSAFE MODULE I suspect this was intended to explode that relaxed restriction. Also probable a C weak semantics could make things worse or better than they are, so ... Thanks in advance --- El lun, 27/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] race conditions in lock-free code... Para: "Jay K" CC: m3devel at elegosoft.com Fecha: lunes, 27 de agosto, 2012 13:34 yeah I agree. processor 1 performs... x? := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent.? If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive.? In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2.? (Or Exclusive, if P1 writes it back and invalidates.)? This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global.? The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice.? I'd suspect that a multiprocessor 21264 could be.? But I do agree it's a bug relative to the Alpha Architecture Manual.? A very nasty one at that. ? ? Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily.? The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way.? It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory.? I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature?? How are t= >he >> > pthreads doing?? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> >? ???Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running.? How is that possible?? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere.? Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run.? Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though.? But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>>? ???Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory.? But is it important?? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR.? = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>>? ? Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>>? ???fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>>???END=3D3D3B >> >>>>>???RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>? ???WITH a =3D3D3D Text8.New (buf) DO >> >>>>>? ? ???MemoryBarrier()=3D3D3B >> >>>>>? ? ???fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>>? ? END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>>? ? ? ? ? ? ? ? ? ? ? ???=3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
? ? ? ? ? ? ? ? ? ? ? ???
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >>? ? ? ? ? ? ? ? ? ? ? ???=3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running.? How is that possible?? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere.? = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run.? Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though.? But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B? ? ? Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory.? But is it important?? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR.? But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B? ???Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B? VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B&g= >t=3D3B? ? IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B? ? ? buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B? ? ? fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? END=3D3D3B
>=3D3B >=3D3B>=3D3= >B? ? RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B? END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B? VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B???=3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? ? WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B? ? ? ? MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B? ? ? ? fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B? ???END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
? ? ? ? ? ? ? ? ? ? ? ???
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > ??? ???????? ?????? ??? ? = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily.? The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way.? It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory.? I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature?? How are the
>=3B >=3B pthreads do= >ing?? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B? ???Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible?? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere.? Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run.? Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though.? But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B? ???Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory.? But is it important?? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR.? But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B? ? Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B? ???buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B? ???fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B???END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >? ???buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B? ???WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B???END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B? ? ? ? ? ? ? ? ? ? ???= >? <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning.? How is that possible?? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere.? Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run.? Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though.? But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B? ? ? Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory.? But is it important?? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR.? But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B? ???Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B? VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ???= > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B? BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B???=3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B? ? ? ? MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? ? fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B? ???END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B? ? ? ? ? ? ? ? ? ? ? ???<=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
??? ???????? ?????? ??? ? v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 28 02:40:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 20:40:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > > yeah I agree. > > processor 1 performs... > > x := alloc_new_text > x^ := character > return x > global := x > > processor 2 performs > > IF global # NIL THEN use(global) END > > now if out of all this, x^ := character is held back, e.g., in a processor > write buffer, and if use(global) involves performing global^, the value > of global^ will be the value of global^ just after x := alloc_new_text > and before x^ := character. > > I believe very aggressive architectures such as Alpha permit reordering > to that extent. If memory serves, on Alpha, if you do not perform a > Memory Barrier, pretty much nothing can be inferred on one processor > about the ordering of operations on another. > > X86 I am not sure is that aggressive. In a hardware-synchronized > multiprocessor environment without barrier operations using e.g. the > Illinois/MESI protocol (doesn't X86?), at the point x^ := character is > performed, the cache coherence protocol has to guarantee the cache line > is in state Exclusive, which means it has to be evicted from the cache > in processor 2 and when use(global) comes around the protocol will > have to ensure it is in Shared state on processor 2. (Or Exclusive, > if P1 writes it back and invalidates.) This would guarantee that you > could not actually see a very old value for global^ at the same time > as a newer value for global. The protocol guarantees that the relative > ordering of writes is maintained across processors. > > Of course I don't know whether Alphas are actually that aggressive in > practice. I'd suspect that a multiprocessor 21264 could be. But I do > agree it's a bug relative to the Alpha Architecture Manual. A very nasty > one at that. > > Mika > > > Jay K writes: >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> It is NOT benign. >> The global can be written before the initialization of what it points to is= >> done. >> The result is wrong. >> >> >> It IS benign if multiple threads go and allocate separate multiple initiali= >> zed TEXTs. >> That's not the problem. >> >> >> - Jay >> >> >>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>> From: hosking at cs.purdue.edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>> To: mika at async.caltech.edu >>> =20 >>> So why is this anything other than benign? It's a cache and there's no pr= >> oblem if different threads get different texts. Don't cripple performance u= >> nnecessarily. The collector will reclaim any unreachable text. >>> =20 >>> Sent from my iPad >>> =20 >>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >> wrote: >>> =20 >>>> Oh I see now. >>>> =20 >>>> It's not that the processor reorders in any tricky way. It is that the >>>> write performed by Text8.New could be sitting around processor 1's cach= >> e >>>> indefinitely without actually getting written back to main memory. >>>> Processor 2 might see the new pointer=2C but when reading the memory po= >> inted >>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >> but >>>> maybe not X86 (without further reorderings anyhow)? >>>> =20 >>>> Any chance any of the pthreads bugs could be of this nature? How are t= >> he >>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >> e >>>> system for a while. >>>> =20 >>>> Mika >>>> =20 >>>> =20 >>>> Jay K writes: >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> The processor can reorder. Across call/ret at least in general. >>>>> =20 >>>>> =20 >>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >> ion=3D2C =3D >>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >> "LTO")=3D >>>>> . (We don't=3D2C but real world systems certainly do.) >>>>> =20 >>>>> =20 >>>>> - Jay >>>>> =20 >>>>> =20 >>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >> caltech.=3D >>>>> edu >>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> =3D20 >>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>> =3D20 >>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>> before Text8.New is done running. How is that possible? Surely >>>>>> Text8.New has to calculate its result before it can be assigned >>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>> =3D20 >>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>> right=3D2C though. But it doesn't... >>>>>> =3D20 >>>>>> Am I missing something? >>>>>> =3D20 >>>>>> Mika >>>>>> =3D20 >>>>>> Mika Nystrom writes: >>>>>>> =20 >>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >> you g=3D >>>>> et >>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >> But no >>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >> n >>>>>>> you call it with the same CHAR... >>>>>>> =20 >>>>>>> Mika >>>>>>> =20 >>>>>>> Jay K writes: >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >> * 1-char t=3D >>>>> exts *) >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> BEGIN >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> RETURN fromCharCache [c] >>>>>>>> END FromChar=3D3D3B >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> It should be: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> =3D3D0A=3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> BEGIN >>>>>>>> =3D3D0A=3D3D >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> =3D3D0A=3D3D >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> RETURN fromCharCache [c] >>>>>>>> =3D3D0A=3D3D >>>>>>>> END FromChar=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >> fro=3D >>>>> mCha=3D3D >>>>>>>> rCache[c] is made. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Can the compiler somehow catch these? >>>>>>>> I fear they are a small epidemic. >>>>>>>> For a long time people didn't realize where all the compiler and pr= >> oces=3D >>>>> sor =3D3D >>>>>>>> could reorder. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>> I think we do. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> - Jay >>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>
We have race con= >> ditions=3D >>>>> =3D3D2C like =3D3D >>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >> 3D ARRAY=3D >>>>> CHAR OF=3D3D >>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >> omChar =3D >>>>> (c: CHAR=3D3D >>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B> r> =3D3D=3D >>>>> 3B BEGIN
=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
&=3D >>>>> nbsp=3D3D3B =3D3D >>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >> 3B
 =3D3D3B=3D >>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >> 3D3B
&nbs=3D >>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >> URN fromCharCa=3D >>>>> che [c]
&nb=3D3D >>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >> ROCEDURE=3D >>>>> FromCha=3D3D >>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
=3D >>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >> ] :=3D3D3D c=3D3D3=3D >>>>> B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >> =3D3D3D Text8.N=3D >>>>> ew (buf) DO
&=3D3D >>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B =3D3D3B M=3D >>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >> p=3D3D3B =3D3D=3D >>>>> 3B fromCharCache[c=3D3D >>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B END=3D3D3B>>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >> =3D3D0A=3D3D >>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >> nmen=3D >>>>> t to=3D3D >>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >> th=3D >>>>> ese?=3D3D >>>>>>>>
I fear they are a small epidemic.
For a long time people did= >> n't =3D >>>>> real=3D3D >>>>>>>> ize where all the compiler and processor could reorder.


= >> Do w=3D >>>>> e ha=3D3D >>>>>>>> ve the right constructs by now to fix them?
I think we do.
> r>>>>> r> -=3D3D >>>>>>>> Jay
>>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> >>>>> >>>>> >>>>>
The processor can reord= >> er. Acros=3D >>>>> s call/ret at least in general.


The compiler can also reord= >> er=3D2C=3D >>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >> oft "L=3D >>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >> world sy=3D >>>>> stems certainly do.)


 =3D3B- Jay


> d=3D3D"Sk=3D >>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >> el at elegos=3D >>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >> race con=3D >>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >> :32:48 -0=3D >>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >> Sorry=3D2C I =3D >>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >> t Text8.N=3D >>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >> t8.New =3D >>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >> has to =3D >>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >> Yes pos=3D >>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >> but th=3D >>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >> n until=3D >>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >> runni=3D >>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >> know co=3D >>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>
&g=3D >>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >> be
=3D >>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >> B Am I missin=3D >>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >> 3B Mika Nystr=3D >>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >> ondition=3D2C i=3D >>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >> ong copy=3D >>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >> 3B >=3D3B=3D >>>>> one ever said FromChar was guaranteed to return the same pointer when<= >> br>&g=3D >>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >> br>>=3D3B >=3D >>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >> =3D3B>=3D3BCon=3D >>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >> 3B>=3D3BConte=3D >>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>> >=3D3B >=3D >>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >> c/Text.m3:>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >> t=3D3B>=3D3BVAR fromC=3D >>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >> ts *)
&g=3D >>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3BPROCEDURE Fr=3D >>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >> AY [0..0] OF =3D >>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >> t=3D3B IF fromCh=3D >>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >> :=3D3D3D c=3D3D3=3D >>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >> w (buf)=3D3D3B<=3D >>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >> B RETURN fromCha=3D >>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >> 3B >=3D3B>=3D3B=3D >>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >> :
>=3D3B >=3D >>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >> BPROCEDURE FromChar (=3D >>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B VA=3D >>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >> =3D3D
>=3D3B &=3D >>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >> =3D3B >=3D3B>=3D3B =3D >>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >> D0A=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>> >=3D3B >=3D >>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >> 3B fromCha=3D >>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >> B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B> r>>=3D3B >=3D3B>=3D3B=3D >>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >> =3D3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B
>=3D3B &=3D >>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >> ssignment=3D >>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >> =3D3B >=3D3B>=3D >>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >> compiler somehow c=3D >>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >> ic.
>=3D >>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >> the compile=3D >>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >> t=3D3B >=3D3B&g=3D >>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >> have the right cons=3D >>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >> br>>=3D3B &=3D >>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >> =3D3B - Jay
>=3D3B &=3D >>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >> ent-Type: t=3D >>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >> tent-Transfer-E=3D >>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D >>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >> >=3D3B >=3D3B>=3D3B=3D >>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >> e P
>=3D3B >=3D >>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >> 3B >=3D3B>=3D3Bpadd=3D >>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >> .hmmessage
>=3D >>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >> 3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >> =3D3B >=3D3B>=3D3B=3D >>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D3Bbod=3D >>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >> have race cond=3D >>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >> e/src/Text.m3:<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >> =3D3D3D ARRAY CHAR =3D >>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >> AR=3D3D
>=3D3B=3D >>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >> buf: ARRAY [0..0]=3D >>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >> t=3D3B=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >> Bnbsp=3D3D3B IF fromCha=3D >>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >> =3D3Bnbsp=3D3D
&g=3D >>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B buf =3D >>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >> sp=3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B fromCharCac=3D >>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >> 3B<=3D3Bbr>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >> arCache [c]<=3D3B=3D >>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >> romChar=3D3D3B<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >> t=3D3B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >> =3D3Br (c: CHAR): T =3D >>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >> >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >> =3D3D3D NIL THEN&l=3D >>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >> =3D3D3D c=3D3D3B<=3D >>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >> 3D3D Text8.New (buf=3D >>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B&=3D3Bnbs=3D >>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B&am=3D >>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >> >=3D3B>=3D3B] :=3D >>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >> amp=3D3Bnbsp=3D3D3B RET=3D >>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >> 3B>=3D3B&=3D3Bnb=3D >>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >> B >=3D3B>=3D3B<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >> ishes before =3D >>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >> s made.<=3D3B=3D >>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >> catch these?=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >> small epidemic.&l=3D >>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >> t=3D3B>=3D3Bize=3D >>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >> lt=3D3Bbr&=3D >>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >> the right construct=3D >>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >> B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >> Bbr>=3D3B =3D >>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >> <=3D3B/html>=3D3B=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >> 896e-68ee-44da-82dd=3D >>>>> -311cd8c58979_--
>>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
It is NOT benign.
The global = >> can be written before the initialization of what it points to is done.
T= >> he result is wrong.


It IS benign if multiple threads go and allo= >> cate separate multiple initialized TEXTs.
That's not the problem.
>>
 =3B- Jay


&g= >> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >> r than benign? It's a cache and there's no problem if different threads get= >> different texts. Don't cripple performance unnecessarily. The collector w= >> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >> ly without actually getting written back to main memory.
>=3B >=3B P= >> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >> ensure that all of Text8.New() finishes before the assignment to fro=3D>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >> long time people didn't realize where all the compiler and proces=3D
&g= >> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
> v> >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From jay.krell at cornell.edu Tue Aug 28 04:45:45 2012 From: jay.krell at cornell.edu (Jay) Date: Mon, 27 Aug 2012 19:45:45 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. > > Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. > > It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile +1 765 427 5484 > > > > > > On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > >> >> yeah I agree. >> >> processor 1 performs... >> >> x := alloc_new_text >> x^ := character >> return x >> global := x >> >> processor 2 performs >> >> IF global # NIL THEN use(global) END >> >> now if out of all this, x^ := character is held back, e.g., in a processor >> write buffer, and if use(global) involves performing global^, the value >> of global^ will be the value of global^ just after x := alloc_new_text >> and before x^ := character. >> >> I believe very aggressive architectures such as Alpha permit reordering >> to that extent. If memory serves, on Alpha, if you do not perform a >> Memory Barrier, pretty much nothing can be inferred on one processor >> about the ordering of operations on another. >> >> X86 I am not sure is that aggressive. In a hardware-synchronized >> multiprocessor environment without barrier operations using e.g. the >> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >> performed, the cache coherence protocol has to guarantee the cache line >> is in state Exclusive, which means it has to be evicted from the cache >> in processor 2 and when use(global) comes around the protocol will >> have to ensure it is in Shared state on processor 2. (Or Exclusive, >> if P1 writes it back and invalidates.) This would guarantee that you >> could not actually see a very old value for global^ at the same time >> as a newer value for global. The protocol guarantees that the relative >> ordering of writes is maintained across processors. >> >> Of course I don't know whether Alphas are actually that aggressive in >> practice. I'd suspect that a multiprocessor 21264 could be. But I do >> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >> one at that. >> >> Mika >> >> >> Jay K writes: >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> It is NOT benign. >>> The global can be written before the initialization of what it points to is= >>> done. >>> The result is wrong. >>> >>> >>> It IS benign if multiple threads go and allocate separate multiple initiali= >>> zed TEXTs. >>> That's not the problem. >>> >>> >>> - Jay >>> >>> >>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>> From: hosking at cs.purdue.edu >>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>> To: mika at async.caltech.edu >>>> =20 >>>> So why is this anything other than benign? It's a cache and there's no pr= >>> oblem if different threads get different texts. Don't cripple performance u= >>> nnecessarily. The collector will reclaim any unreachable text. >>>> =20 >>>> Sent from my iPad >>>> =20 >>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>> wrote: >>>> =20 >>>>> Oh I see now. >>>>> =20 >>>>> It's not that the processor reorders in any tricky way. It is that the >>>>> write performed by Text8.New could be sitting around processor 1's cach= >>> e >>>>> indefinitely without actually getting written back to main memory. >>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>> inted >>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>> but >>>>> maybe not X86 (without further reorderings anyhow)? >>>>> =20 >>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>> he >>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>> e >>>>> system for a while. >>>>> =20 >>>>> Mika >>>>> =20 >>>>> =20 >>>>> Jay K writes: >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> The processor can reorder. Across call/ret at least in general. >>>>>> =20 >>>>>> =20 >>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>> ion=3D2C =3D >>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>> "LTO")=3D >>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>> =20 >>>>>> =20 >>>>>> - Jay >>>>>> =20 >>>>>> =20 >>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>> caltech.=3D >>>>>> edu >>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>> From: mika at async.caltech.edu >>>>>>> =3D20 >>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>> =3D20 >>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>> =3D20 >>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>> right=3D2C though. But it doesn't... >>>>>>> =3D20 >>>>>>> Am I missing something? >>>>>>> =3D20 >>>>>>> Mika >>>>>>> =3D20 >>>>>>> Mika Nystrom writes: >>>>>>>> =20 >>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>> you g=3D >>>>>> et >>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>> But no >>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>> n >>>>>>>> you call it with the same CHAR... >>>>>>>> =20 >>>>>>>> Mika >>>>>>>> =20 >>>>>>>> Jay K writes: >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>> * 1-char t=3D >>>>>> exts *) >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> BEGIN >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> It should be: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> BEGIN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>> fro=3D >>>>>> mCha=3D3D >>>>>>>>> rCache[c] is made. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Can the compiler somehow catch these? >>>>>>>>> I fear they are a small epidemic. >>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>> oces=3D >>>>>> sor =3D3D >>>>>>>>> could reorder. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>> I think we do. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> - Jay >>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>
We have race con= >>> ditions=3D >>>>>> =3D3D2C like =3D3D >>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>> 3D ARRAY=3D >>>>>> CHAR OF=3D3D >>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>> omChar =3D >>>>>> (c: CHAR=3D3D >>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>> r> =3D3D=3D >>>>>> 3B BEGIN
=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
&=3D >>>>>> nbsp=3D3D3B =3D3D >>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>> 3B
 =3D3D3B=3D >>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>> 3D3B
&nbs=3D >>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>> URN fromCharCa=3D >>>>>> che [c]
&nb=3D3D >>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>> ROCEDURE=3D >>>>>> FromCha=3D3D >>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
=3D >>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>> ] :=3D3D3D c=3D3D3=3D >>>>>> B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>> =3D3D3D Text8.N=3D >>>>>> ew (buf) DO
&=3D3D >>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B =3D3D3B M=3D >>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>> p=3D3D3B =3D3D=3D >>>>>> 3B fromCharCache[c=3D3D >>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B END=3D3D3B>>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>> nmen=3D >>>>>> t to=3D3D >>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>> th=3D >>>>>> ese?=3D3D >>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>> n't =3D >>>>>> real=3D3D >>>>>>>>> ize where all the compiler and processor could reorder.


= >>> Do w=3D >>>>>> e ha=3D3D >>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>> r>>>>>> r> -=3D3D >>>>>>>>> Jay
>>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> >>>>>> >>>>>> >>>>>>
The processor can reord= >>> er. Acros=3D >>>>>> s call/ret at least in general.


The compiler can also reord= >>> er=3D2C=3D >>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>> oft "L=3D >>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>> world sy=3D >>>>>> stems certainly do.)


 =3D3B- Jay


>> d=3D3D"Sk=3D >>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>> el at elegos=3D >>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>> race con=3D >>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>> :32:48 -0=3D >>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>> Sorry=3D2C I =3D >>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>> t Text8.N=3D >>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>> t8.New =3D >>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>> has to =3D >>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>> Yes pos=3D >>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>> but th=3D >>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>> n until=3D >>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>> runni=3D >>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>> know co=3D >>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>
&g=3D >>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>> be
=3D >>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>> B Am I missin=3D >>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>> 3B Mika Nystr=3D >>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>> ondition=3D2C i=3D >>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>> ong copy=3D >>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>> 3B >=3D3B=3D >>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>> br>&g=3D >>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>> br>>=3D3B >=3D >>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>> =3D3B>=3D3BCon=3D >>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>> 3B>=3D3BConte=3D >>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>> >=3D3B >=3D >>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>> c/Text.m3:>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>> t=3D3B>=3D3BVAR fromC=3D >>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>> ts *)
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>> AY [0..0] OF =3D >>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>> t=3D3B IF fromCh=3D >>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>> :=3D3D3D c=3D3D3=3D >>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>> w (buf)=3D3D3B<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>> B RETURN fromCha=3D >>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3B=3D >>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>> :
>=3D3B >=3D >>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>> BPROCEDURE FromChar (=3D >>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B VA=3D >>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>> =3D3D
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>> =3D3B >=3D3B>=3D3B =3D >>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>> D0A=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>> >=3D3B >=3D >>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>> 3B fromCha=3D >>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>> B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>> =3D3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B
>=3D3B &=3D >>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>> ssignment=3D >>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>> =3D3B >=3D3B>=3D >>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>> compiler somehow c=3D >>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>> ic.
>=3D >>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>> the compile=3D >>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>> t=3D3B >=3D3B&g=3D >>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>> have the right cons=3D >>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>> br>>=3D3B &=3D >>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>> =3D3B - Jay
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>> ent-Type: t=3D >>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>> tent-Transfer-E=3D >>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D >>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>> >=3D3B >=3D3B>=3D3B=3D >>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>> e P
>=3D3B >=3D >>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3Bpadd=3D >>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>> .hmmessage
>=3D >>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>> 3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>> =3D3B >=3D3B>=3D3B=3D >>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>> have race cond=3D >>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>> e/src/Text.m3:<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>> =3D3D3D ARRAY CHAR =3D >>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>> AR=3D3D
>=3D3B=3D >>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>> buf: ARRAY [0..0]=3D >>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>> t=3D3B=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>> Bnbsp=3D3D3B IF fromCha=3D >>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>> =3D3Bnbsp=3D3D
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B fromCharCac=3D >>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>> 3B<=3D3Bbr>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>> arCache [c]<=3D3B=3D >>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>> romChar=3D3D3B<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>> t=3D3B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>> =3D3Br (c: CHAR): T =3D >>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>> >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>> =3D3D3D NIL THEN&l=3D >>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>> =3D3D3D c=3D3D3B<=3D >>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>> 3D3D Text8.New (buf=3D >>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>> >=3D3B>=3D3B] :=3D >>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>> 3B>=3D3B&=3D3Bnb=3D >>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>> B >=3D3B>=3D3B<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>> ishes before =3D >>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>> s made.<=3D3B=3D >>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>> catch these?=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>> small epidemic.&l=3D >>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>> t=3D3B>=3D3Bize=3D >>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>> lt=3D3Bbr&=3D >>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>> the right construct=3D >>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>> B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>> Bbr>=3D3B =3D >>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>> <=3D3B/html>=3D3B=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>> 896e-68ee-44da-82dd=3D >>>>>> -311cd8c58979_--
>>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/html; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> >>> >>> >>>
It is NOT benign.
The global = >>> can be written before the initialization of what it points to is done.
T= >>> he result is wrong.


It IS benign if multiple threads go and allo= >>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>
 =3B- Jay


&g= >>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>> r than benign? It's a cache and there's no problem if different threads get= >>> different texts. Don't cripple performance unnecessarily. The collector w= >>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>> ly without actually getting written back to main memory.
>=3B >=3B P= >>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>> long time people didn't realize where all the compiler and proces=3D
&g= >>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>> v> >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- > From hosking at cs.purdue.edu Tue Aug 28 05:21:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 23:21:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: <80443A80-9E57-4994-B0EF-B31C5FC98EB5@cs.purdue.edu> Indeed, a lock is overkill. In general any constructor (initializer) will need a StoreStore barrier (see http://g.oswego.edu/dl/jmm/cookbook.html) before returning a pointer to the constructed storage. Sent from my iPad On Aug 27, 2012, at 10:45 PM, Jay wrote: > Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. SPARC is TSO. So not necessary. StoreStore = nop. ARM needs dmb-st. Alpha needs wmb. I don't know what a StoreStore barrier looks like on MIPS. > > A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. > > Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... Overkill. Don't scare the horses unnecessarily. Concurrent programmers must always beware. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > >> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. >> >> Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. >> >> It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. >> >> Antony Hosking | Associate Professor | Computer Science | Purdue University >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Mobile +1 765 427 5484 >> >> >> >> >> >> On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: >> >>> >>> yeah I agree. >>> >>> processor 1 performs... >>> >>> x := alloc_new_text >>> x^ := character >>> return x >>> global := x >>> >>> processor 2 performs >>> >>> IF global # NIL THEN use(global) END >>> >>> now if out of all this, x^ := character is held back, e.g., in a processor >>> write buffer, and if use(global) involves performing global^, the value >>> of global^ will be the value of global^ just after x := alloc_new_text >>> and before x^ := character. >>> >>> I believe very aggressive architectures such as Alpha permit reordering >>> to that extent. If memory serves, on Alpha, if you do not perform a >>> Memory Barrier, pretty much nothing can be inferred on one processor >>> about the ordering of operations on another. >>> >>> X86 I am not sure is that aggressive. In a hardware-synchronized >>> multiprocessor environment without barrier operations using e.g. the >>> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >>> performed, the cache coherence protocol has to guarantee the cache line >>> is in state Exclusive, which means it has to be evicted from the cache >>> in processor 2 and when use(global) comes around the protocol will >>> have to ensure it is in Shared state on processor 2. (Or Exclusive, >>> if P1 writes it back and invalidates.) This would guarantee that you >>> could not actually see a very old value for global^ at the same time >>> as a newer value for global. The protocol guarantees that the relative >>> ordering of writes is maintained across processors. >>> >>> Of course I don't know whether Alphas are actually that aggressive in >>> practice. I'd suspect that a multiprocessor 21264 could be. But I do >>> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >>> one at that. >>> >>> Mika >>> >>> >>> Jay K writes: >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> It is NOT benign. >>>> The global can be written before the initialization of what it points to is= >>>> done. >>>> The result is wrong. >>>> >>>> >>>> It IS benign if multiple threads go and allocate separate multiple initiali= >>>> zed TEXTs. >>>> That's not the problem. >>>> >>>> >>>> - Jay >>>> >>>> >>>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>>> From: hosking at cs.purdue.edu >>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>>> To: mika at async.caltech.edu >>>>> =20 >>>>> So why is this anything other than benign? It's a cache and there's no pr= >>>> oblem if different threads get different texts. Don't cripple performance u= >>>> nnecessarily. The collector will reclaim any unreachable text. >>>>> =20 >>>>> Sent from my iPad >>>>> =20 >>>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>>> wrote: >>>>> =20 >>>>>> Oh I see now. >>>>>> =20 >>>>>> It's not that the processor reorders in any tricky way. It is that the >>>>>> write performed by Text8.New could be sitting around processor 1's cach= >>>> e >>>>>> indefinitely without actually getting written back to main memory. >>>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>>> inted >>>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>>> but >>>>>> maybe not X86 (without further reorderings anyhow)? >>>>>> =20 >>>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>>> he >>>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>>> e >>>>>> system for a while. >>>>>> =20 >>>>>> Mika >>>>>> =20 >>>>>> =20 >>>>>> Jay K writes: >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> The processor can reorder. Across call/ret at least in general. >>>>>>> =20 >>>>>>> =20 >>>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>>> ion=3D2C =3D >>>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>>> "LTO")=3D >>>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>>> =20 >>>>>>> =20 >>>>>>> - Jay >>>>>>> =20 >>>>>>> =20 >>>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>>> caltech.=3D >>>>>>> edu >>>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>>> From: mika at async.caltech.edu >>>>>>>> =3D20 >>>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>>> =3D20 >>>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>>> =3D20 >>>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>>> right=3D2C though. But it doesn't... >>>>>>>> =3D20 >>>>>>>> Am I missing something? >>>>>>>> =3D20 >>>>>>>> Mika >>>>>>>> =3D20 >>>>>>>> Mika Nystrom writes: >>>>>>>>> =20 >>>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>>> you g=3D >>>>>>> et >>>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>>> But no >>>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>>> n >>>>>>>>> you call it with the same CHAR... >>>>>>>>> =20 >>>>>>>>> Mika >>>>>>>>> =20 >>>>>>>>> Jay K writes: >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>>> * 1-char t=3D >>>>>>> exts *) >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> BEGIN >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> It should be: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> BEGIN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>>> fro=3D >>>>>>> mCha=3D3D >>>>>>>>>> rCache[c] is made. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Can the compiler somehow catch these? >>>>>>>>>> I fear they are a small epidemic. >>>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>>> oces=3D >>>>>>> sor =3D3D >>>>>>>>>> could reorder. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>>> I think we do. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> - Jay >>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>
We have race con= >>>> ditions=3D >>>>>>> =3D3D2C like =3D3D >>>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>>> 3D ARRAY=3D >>>>>>> CHAR OF=3D3D >>>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>>> omChar =3D >>>>>>> (c: CHAR=3D3D >>>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>>> r> =3D3D=3D >>>>>>> 3B BEGIN
=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
&=3D >>>>>>> nbsp=3D3D3B =3D3D >>>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>>> 3B
 =3D3D3B=3D >>>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>>> 3D3B
&nbs=3D >>>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>>> URN fromCharCa=3D >>>>>>> che [c]
&nb=3D3D >>>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>>> ROCEDURE=3D >>>>>>> FromCha=3D3D >>>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
=3D >>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>>> ] :=3D3D3D c=3D3D3=3D >>>>>>> B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>>> =3D3D3D Text8.N=3D >>>>>>> ew (buf) DO
&=3D3D >>>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B =3D3D3B M=3D >>>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>>> p=3D3D3B =3D3D=3D >>>>>>> 3B fromCharCache[c=3D3D >>>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B END=3D3D3B>>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>>> nmen=3D >>>>>>> t to=3D3D >>>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>>> th=3D >>>>>>> ese?=3D3D >>>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>>> n't =3D >>>>>>> real=3D3D >>>>>>>>>> ize where all the compiler and processor could reorder.


= >>>> Do w=3D >>>>>>> e ha=3D3D >>>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>>> r>>>>>>> r> -=3D3D >>>>>>>>>> Jay
>>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> >>>>>>> >>>>>>> >>>>>>>
The processor can reord= >>>> er. Acros=3D >>>>>>> s call/ret at least in general.


The compiler can also reord= >>>> er=3D2C=3D >>>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>>> oft "L=3D >>>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>>> world sy=3D >>>>>>> stems certainly do.)


 =3D3B- Jay


>>> d=3D3D"Sk=3D >>>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>>> el at elegos=3D >>>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>>> race con=3D >>>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>>> :32:48 -0=3D >>>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>>> Sorry=3D2C I =3D >>>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>>> t Text8.N=3D >>>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>>> t8.New =3D >>>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>>> has to =3D >>>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>>> Yes pos=3D >>>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>>> but th=3D >>>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>>> n until=3D >>>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>>> runni=3D >>>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>>> know co=3D >>>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>>
&g=3D >>>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>>> be
=3D >>>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>>> B Am I missin=3D >>>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>>> 3B Mika Nystr=3D >>>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>>> ondition=3D2C i=3D >>>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>>> ong copy=3D >>>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>>> 3B >=3D3B=3D >>>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>>> br>&g=3D >>>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>>> =3D3B>=3D3BCon=3D >>>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>>> 3B>=3D3BConte=3D >>>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>>> c/Text.m3:>>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>>> t=3D3B>=3D3BVAR fromC=3D >>>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>>> ts *)
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>>> AY [0..0] OF =3D >>>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>>> t=3D3B IF fromCh=3D >>>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>>> :=3D3D3D c=3D3D3=3D >>>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>>> w (buf)=3D3D3B<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>>> B RETURN fromCha=3D >>>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3B=3D >>>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>>> :
>=3D3B >=3D >>>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>>> BPROCEDURE FromChar (=3D >>>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B VA=3D >>>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>>> =3D3D
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>>> =3D3B >=3D3B>=3D3B =3D >>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>>> D0A=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>>> 3B fromCha=3D >>>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>>> B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>>> =3D3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>>> ssignment=3D >>>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>>> =3D3B >=3D3B>=3D >>>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>>> compiler somehow c=3D >>>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>>> ic.
>=3D >>>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>>> the compile=3D >>>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>>> t=3D3B >=3D3B&g=3D >>>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>>> have the right cons=3D >>>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>>> br>>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>>> =3D3B - Jay
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>>> ent-Type: t=3D >>>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>>> tent-Transfer-E=3D >>>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D >>>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>>> >=3D3B >=3D3B>=3D3B=3D >>>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>>> e P
>=3D3B >=3D >>>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3Bpadd=3D >>>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>>> .hmmessage
>=3D >>>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>>> 3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>>> =3D3B >=3D3B>=3D3B=3D >>>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>>> have race cond=3D >>>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>>> e/src/Text.m3:<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>>> =3D3D3D ARRAY CHAR =3D >>>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>>> AR=3D3D
>=3D3B=3D >>>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>>> buf: ARRAY [0..0]=3D >>>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>>> t=3D3B=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>>> Bnbsp=3D3D3B IF fromCha=3D >>>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>>> =3D3Bnbsp=3D3D
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B fromCharCac=3D >>>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>>> 3B<=3D3Bbr>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>>> arCache [c]<=3D3B=3D >>>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>>> romChar=3D3D3B<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>>> t=3D3B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>>> =3D3Br (c: CHAR): T =3D >>>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>>> >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>>> =3D3D3D NIL THEN&l=3D >>>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>>> =3D3D3D c=3D3D3B<=3D >>>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>>> 3D3D Text8.New (buf=3D >>>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>>> >=3D3B>=3D3B] :=3D >>>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>>> 3B>=3D3B&=3D3Bnb=3D >>>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>>> B >=3D3B>=3D3B<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>>> ishes before =3D >>>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>>> s made.<=3D3B=3D >>>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>>> catch these?=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>>> small epidemic.&l=3D >>>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>>> t=3D3B>=3D3Bize=3D >>>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>>> lt=3D3Bbr&=3D >>>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>>> the right construct=3D >>>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>>> B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>>> Bbr>=3D3B =3D >>>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>>> <=3D3B/html>=3D3B=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>>> 896e-68ee-44da-82dd=3D >>>>>>> -311cd8c58979_--
>>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/html; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>>
It is NOT benign.
The global = >>>> can be written before the initialization of what it points to is done.
T= >>>> he result is wrong.


It IS benign if multiple threads go and allo= >>>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>>
 =3B- Jay


&g= >>>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>>> r than benign? It's a cache and there's no problem if different threads get= >>>> different texts. Don't cripple performance unnecessarily. The collector w= >>>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>>> ly without actually getting written back to main memory.
>=3B >=3B P= >>>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>>> long time people didn't realize where all the compiler and proces=3D
&g= >>>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>>> v> >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- >> From dabenavidesd at yahoo.es Wed Aug 29 18:08:28 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 29 Aug 2012 17:08:28 +0100 (BST) Subject: [M3devel] Copyright clearance in Modula-3 books Message-ID: <1346256508.9030.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: how about asking for any part involved copyright clearance (compliance) with respect of SPwM3, M3 and AiM3. I knew of recent efforts here: http://thid.thesa.com/thid-0698-8201-th-1129-8613 Shouldn't we ask Greg Nelson for that? In that case we wouldn't need direct authorization to publish the Book, but just that, else stay tuned for a licensing of the material for us. Anyway if we have the sources of the book and nobody's claim that, couldn't be already cleared? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 2 09:28:40 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Message-ID: Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 2 09:38:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 2 Aug 2012 07:38:52 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: References: Message-ID: okok..4.5 generates: ldx [%fp+2039], %g1 and %g1, 7, %g1 and %g1, 0xff, %g1 cmp %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: andcc %i0, 7, %g0 bne,pn %xcc, .LL2 - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89) start_call_indirect type:addr (90) load_integer type:int_64 0 (91) pop_param type:int_64 (92) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93) loophole type1:addr type2:int_64 (94) load_integer type:int_64 7 (95) and type:int_64 (96) if_true type:int_64 label:1 frequency:0X32(50) (97) load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98) load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: save %sp, -208, %sp .LLCFI0: stx %i0, [%fp+2175] ldx [%fp+2175], %g1 stx %g1, [%fp+2039] ldx [%fp+2039], %g1 and %g1, 7, %g1 <= correct and %g1, 1, %g1 <= incorrect and %g1, 0xff, %g1 <= pointless, inefficient cmp %g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); EXPR_POP (); add_stmt (build3 (COND_EXPR, t_void, cond, build1 (GOTO_EXPR, t_void, label), NULL_TREE)); } static tree m3_cast (tree type, tree op0) { return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val; true, false: Label; freq: Frequency) = VAR skip := Next_label (); nope := skip; BEGIN IF (false # No_label) THEN nope := false; END; IF NOT Target.Aligned_procedures THEN Push (proc); Force (); cg.loophole (Type.Addr, Target.Integer.cg_type); Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); cg.and (Target.Integer.cg_type); == insert comparisong to 0 here == cg.if_true (Target.Integer.cg_type, nope, Always - freq); SPop (1, "If_closure-unaligned"); END; and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 2 17:36:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 2 Aug 2012 16:36:40 +0100 (BST) Subject: [M3devel] unaligned procedures closure marker check broken? In-Reply-To: Message-ID: <1343921800.38422.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: but what pointers are you trying to diagnose, if it's a function closure then a REF to a RECORD paired with PROCEDURE is, that is, an OBJECT, but function "tagged" types, I don't think are M3language elements, that doesn't make them BRANDED by default, but structural equivalent by default just different for each type of procedure or method, if you think so, many languages have them, included C I guess, but why do we need those to build the compiler itself, perhaps object oriented environments can look for those in question, you can look for a hack B Kallsow tried to do in m3browser with RTType but I guess for structural types not branded types, so perhaps we need them both, it could be a nice work to do in static time, DEC experimented with some HW as a target for Cardelli's typeful Quest language, we could ask more about that experiment, since it targeted Modules static info and we could add everything you want in that. I need more time to do so, but looks interesting to do. Thanks in advance --- El jue, 2/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] unaligned procedures closure marker check broken? Para: "m3devel" Fecha: jueves, 2 de agosto, 2012 02:38 okok..4.5 generates: ??? ldx??? [%fp+2039], %g1 ??? and??? %g1, 7, %g1 ??? and??? %g1, 0xff, %g1 ??? cmp??? %g1, 0 which is good. The and 0xff is still pointless. This is without optimizing. optimized it is: ??? andcc??? %i0, 7, %g0 ??? bne,pn??? %xcc, .LL2 ?- Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Thu, 2 Aug 2012 07:28:40 +0000 Subject: [M3devel] unaligned procedures closure marker check broken? Given: MODULE RTTest; IMPORT RT0; PROCEDURE F1 (b: RT0.Binder) = BEGIN EVAL b(0); END F1; BEGIN END RTTest. This tends to fail on Aligned_procedures := FALSE architectures. I see this testing SPARC64_SOLARIS with gcc 4.7 backend, when the procedure isn't 8-aligned -- alignment fault. IL is reasonable: ... (89)???????? start_call_indirect type:addr (90)?????????? load_integer type:int_64 0 (91)?????????? pop_param type:int_64 (92)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (93)?????????? loophole type1:addr type2:int_64 (94)?????????? load_integer type:int_64 7 (95)?????????? and type:int_64 (96)?????????? if_true type:int_64 label:1 frequency:0X32(50) (97)?????????? load var:0x5 offset:0 src_t:addr dst_t:addr m3name:noname (98)?????????? load_nil ... but the assembly is inefficient and incorrect: RTTest__F1: .LLFB0: ?? ?save?? ?%sp, -208, %sp .LLCFI0: ?? ?stx?? ?%i0, [%fp+2175] ?? ?ldx?? ?[%fp+2175], %g1 ?? ?stx?? ?%g1, [%fp+2039] ?? ?ldx?? ?[%fp+2039], %g1 ?? ?and?? ?%g1, 7, %g1 <= correct ?? ?and?? ?%g1, 1, %g1? <= incorrect ?? ?and?? ?%g1, 0xff, %g1? <=? pointless, inefficient ?? ?cmp?? ?%g1, 0 current parse.c: M3CG_HANDLER (IF_TRUE) { ? tree cond = m3_cast (boolean_type_node, EXPR_REF (-1)); ? EXPR_POP (); ? add_stmt (build3 (COND_EXPR, t_void, cond, ??????????????????? build1 (GOTO_EXPR, t_void, label), ??????????????????? NULL_TREE)); } static tree m3_cast (tree type, tree op0) { ? return m3_build1 (NOP_EXPR, type, op0); } Thoughts? I'm thinking, something like: PROCEDURE If_closure (proc: Val;? true, false: Label;? freq: Frequency) = ? VAR skip := Next_label ();? nope := skip; ? BEGIN ??? IF (false # No_label) THEN? nope := false; END; ??? IF NOT Target.Aligned_procedures THEN ????? Push (proc); ????? Force (); ????? cg.loophole (Type.Addr, Target.Integer.cg_type); ????? Push_int (TargetMap.CG_Align_bytes[Target.Integer.cg_type] - 1); ????? cg.and (Target.Integer.cg_type); ???? ? ????? == insert comparisong to 0 here == ????? cg.if_true (Target.Integer.cg_type, nope, Always - freq); ????? SPop (1, "If_closure-unaligned"); ??? END; ?? ? and possibly if_true should assert that top of stack is boolean already. I'll check that this isn't something I broke in 4.7 -- maybe boolean changed. I was thinking this had something to do with tagged types, but no -- there is nothing related to them in the IL. I guess function pointers can't be tagged? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 6 05:48:22 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 03:48:22 +0000 Subject: [M3devel] addresses of constants? Message-ID: Ugzip.m3 does an unnecessary heap allocation and copy. Here I have changed it to avoid the heap allocation. But it still makes a copy. How can I fix that? It seems ADR cannot be applied to constants? CONST ZLIB_VERSION = ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; This works but still is inefficient: PROCEDURE deflateInit(strm: z_stream_star; level: int): int = VAR version := ZLIB_VERSION; BEGIN RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); END deflateInit; I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 6 19:40:21 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 06 Aug 2012 12:40:21 -0500 Subject: [M3devel] addresses of constants? In-Reply-To: References: Message-ID: <50200185.1060106@lcwb.coop> You probably already know this, but, from 2.7: ADR (VAR x: Any) : ADDRESS ... ADR(x) is the address of the variable x. The actual argument must be a designator but need not be writable. And from 2.6.3: An identifier is a writable designator if it is declared as a variable, is a VAR or VALUE parameter, is a local of a TYPECASE or TRY-EXCEPT statement, or is a WITH local that is bound to a writable designator. An identifier is a readonly designator if it is a READONLY parameter, a local of a FOR statement, or a WITH local bound to a nondesignator or readonly designator. I'm speculating, but confidently, that the reason for the designator restriction is to have an implementation-independent rule that ensures the value would be in memory, and thus have an address, regardless of whether ADR is applied to it or not, The ADR could happen in a separate compilation from the declaration, and its existence be unknown at least until the main program was compiled. This way, compiler writers have the option of putting any constant in immediate instruction fields, if the target machine has them, without risk of undermining source code that compiled and ran fine on a different machine or with a different compiler. Even if it's sure to be in memory, allowing ADR(SomeConstant) also would open up the possibility of a constant that wasn't constant, a la FORTRAN. Yeah, I know it's already unsafe code anyway, but that would be an additional form of unsafety with scarcely an imaginable benefit. I'd just make it a VAR and put it inside a module. You would only have to verify that that one module didn't change it. It's possible a smart enough compiler would initialize right in the variable in the executable and not have to copy it. BTW, VAR ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) appears to have another problem. It assumes the representation of "1.0.4" is the same as, or at least contains ARRAY [0..5] OF char{ ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; which it also looks like your modified UgzipP.deflateInit would expect. This happens to be true in pm3, but not cm3. Without debating the merits of various representations of TEXT literals, it would make for implementation-dependent code in any case. On 08/05/2012 10:48 PM, Jay K wrote: > Ugzip.m3 does an unnecessary heap allocation and copy. > > > Here I have changed it to avoid the heap allocation. > But it still makes a copy. > > How can I fix that? > > It seems ADR cannot be applied to constants? > > CONST > ZLIB_VERSION = ARRAY [0..5] OF char{ > ORD('1'), ORD('.'), ORD('0'), ORD('.'), ORD('4'), 0}; > > This works but still is inefficient: > > PROCEDURE deflateInit(strm: z_stream_star; level: int): int = > VAR version := ZLIB_VERSION; > BEGIN > RETURN UgzipP.deflateInit_(strm, level, ADR(version[0]), BYTESIZE(z_stream)); > END deflateInit; > > > I'd like to say ADR(ZLIB_VERSION[0]) or ADR(ZLIB_VERSION); > const ZLIB_VERSION = "1.0.4"; and ADR(ZLIB_VERSION[0]) of that. > > > - Jay From jay.krell at cornell.edu Tue Aug 7 01:02:16 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Aug 2012 23:02:16 +0000 Subject: [M3devel] initializer problem? In-Reply-To: References: <20120806105804.20F9F2474003@birch.elegosoft.com>, , , Message-ID: I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: DepthArray = REF ARRAY OF RECORD n := 0 END; dcount := NEW(DepthArray, 10); FOR i := FIRST(dcount^) TO LAST(dcount^) DO <* ASSERT dcount[i].n = 0 *> END; the assertion fails sometimes. I'll have to keep digging. - Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3 17 Mar 2008 03:46:14 -0000 1.2 +++ VBTRep.m3 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ (* All d: dcount[d].n = # VBTs in list with depth at most d. *) WITH v = NEW(REF ARRAY OF VBT.T, n) DO FOR i := 0 TO n - 1 DO + IF list[i].depth > 0 THEN v[dcount[list[i].depth - 1].n] := list[i].v; INC(dcount[list[i].depth - 1].n); (* All d: dcount[d-1].n = # VBTs in list with depth < d, or with depth = d that have been copied into v. *) END; + END; (* v has all the VBTs in the list and is sorted by depth *) FOR i := 0 TO n - 1 DO IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Tue Aug 7 15:44:27 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:44:27 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211BBB.4070203@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From mfleischer at elego.de Tue Aug 7 15:51:25 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Tue, 07 Aug 2012 09:51:25 -0400 Subject: [M3devel] [cm3 trac maintenance notification] Message-ID: <50211D5D.2040901@elego.de> Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Tue Aug 7 16:28:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 15:28:27 +0100 (BST) Subject: [M3devel] [cm3 trac maintenance notification] In-Reply-To: <50211BBB.4070203@elego.de> Message-ID: <1344349707.21021.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi: how much time, will be down, or just instantly be migrated? Well my hope is all bugs were gone but ... Thanks in advance --- El mar, 7/8/12, Maurizio Fleischer escribi?: De: Maurizio Fleischer Asunto: [M3devel] [cm3 trac maintenance notification] Para: m3devel at elego.de, m3announce at elego.de CC: admin at elego.de Fecha: martes, 7 de agosto, 2012 08:44 Hello, due to maintenance work the cm3 trac project will be down tomorrow 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users accessing https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make your important changes beforehand and log out. We apologize for the inconvenience. Thanks elego systems administration -- Maurizio Fleischer IT Services&? Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96? ? ? maurizio.fleischer at elegosoft.com fax???+49 30 23 45 86 95http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 7 19:50:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 7 Aug 2012 18:50:31 +0100 (BST) Subject: [M3devel] initializer problem? In-Reply-To: Message-ID: <1344361831.31130.YahooMailClassic@web29701.mail.ird.yahoo.com> Hi all: interestingly, Greg Nelson mentioned here: http://books.google.com.co/books?id=EBW2AAAAIAAJ&q=trestle+%22dec-SRC%22&dq=trestle+%22dec-SRC%22&source=bl&ots=RTsXmdey5q&sig=XENEwmkvSMW3BzyeN9w_68VdcHU&hl=en&redir_esc=y that Trestle had some problem in RISC architectures, I wonder what he was referring to, and if they were working in a new design what did they changed. Thanks in advance --- El lun, 6/8/12, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] initializer problem? Para: "m3devel" Fecha: lunes, 6 de agosto, 2012 18:02 I see this fail with the 4.5.x compiler also. FreeBSD/x86. Single proc VM. FreeBSD 9.0 I think. Remoted to Darwin. I'll try to look at it more.. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 11:17:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Sorry, that isn't the right fix. The problem can be better understood like so: ? DepthArray = REF ARRAY OF RECORD n := 0 END; ????? dcount := NEW(DepthArray, 10); ????? FOR i := FIRST(dcount^) TO LAST(dcount^) DO ??????? <* ASSERT dcount[i].n = 0 *> ????? END; the assertion fails sometimes. I'll have to keep digging. ?- Jay From: jay.krell at cornell.edu To: jkrell at elego.de; m3commit at elegosoft.com Date: Mon, 6 Aug 2012 10:58:50 +0000 Subject: Re: [M3commit] CVS Update: cm3 Index: VBTRep.m3 =================================================================== RCS file: /usr/cvs/cm3/m3-ui/ui/src/vbt/VBTRep.m3,v retrieving revision 1.2 diff -u -w -r1.2 VBTRep.m3 --- VBTRep.m3??? 17 Mar 2008 03:46:14 -0000??? 1.2 +++ VBTRep.m3??? 6 Aug 2012 10:56:01 -0000 @@ -641,11 +641,13 @@ ?????? (* All d: dcount[d].n = # VBTs in list with depth at most d. *) ?????? WITH v = NEW(REF ARRAY OF VBT.T, n) DO ???????? FOR i := 0 TO n - 1 DO +????????? IF list[i].depth > 0 THEN ?????????? v[dcount[list[i].depth - 1].n] := list[i].v; ?????????? INC(dcount[list[i].depth - 1].n); ?????????? (* All d: dcount[d-1].n = # VBTs in list with depth < d, ????????????? or with depth = d that have been copied into v. *) ???????? END; +??????? END; ???????? (* v has all the VBTs in the list and is sorted by depth *) ???????? FOR i := 0 TO n - 1 DO ?????????? IF Prop.Marked IN v[i].props THEN > Date: Mon, 6 Aug 2012 12:58:03 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/06 12:58:03 > > Modified files: > cm3/m3-ui/ui/src/vbt/: VBTRep.m3 > > Log message: > avoid subscript out of range at line 644 > test case is starting up most gui apps on FreeBSD/x86 with gcc 4.7 > backend, remoting X to MacOSX 10.5; the distinguishing factor > is probably the relative lack of windows or something funny > with window layering on Mac -- i.e. depth = 0, resulting array > index is -1 and that is invalid > > Trestle experts, please review and think over it more. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfleischer at elego.de Wed Aug 8 16:00:14 2012 From: mfleischer at elego.de (Maurizio Fleischer) Date: Wed, 08 Aug 2012 10:00:14 -0400 Subject: [M3devel] [cm3 trac maintenance notification] project back online In-Reply-To: <50211D5D.2040901@elego.de> References: <50211D5D.2040901@elego.de> Message-ID: <502270EE.80404@elego.de> Hi, the cm3 trac project is online again. As mentioned it is now reachable at https://cm3-bugs.elegosoft.com/cm3/. links in old ticket mails are redirected. kind regards elego systems administration On 08/07/2012 09:51 AM, Maurizio Fleischer wrote: > Hello, > due to maintenance work the cm3 trac project will be down tomorrow > 2012-08-08 at 11:00 am UTC+2. Also it will be moved, so it is > reachable at https://cm3-bugs.elegosoft.com/cm3/ afterwards. Users > accessing > https://bugs.elegosoft.com/cm3/ are then redirected. Be sure to make > your important changes beforehand and log out. > We apologize for the inconvenience. > > Thanks > elego systems administration > -- Maurizio Fleischer IT Services& Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 maurizio.fleischer at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner/Michael Diers, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Sat Aug 11 16:20:40 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:20:40 +0100 (BST) Subject: [M3devel] Elegosoft.com mailer down? Message-ID: <1344694840.84378.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: How may times in a row could I send an malformed email? May I miss something m3devel ? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Aug 11 16:14:41 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 11 Aug 2012 15:14:41 +0100 (BST) Subject: [M3devel] About extended char types, and integer types Message-ID: <1344694481.85507.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: I'm getting serious about this so I hope you don't misunderstand me, see this two differences in point of view of Modula-3 char character type and integer type systems: http://rant.gulbrandsen.priv.no/code/modula3-integers http://slps.svn.sourceforge.net/viewvc/slps/topics/grammars/modula/src-052/src.syntax.txt?view=markup&pathrev=1078 Though in some form, Modula-3 is better than this kind of things today are written,? if somebody takes the time to talk about it, it makes me think that something went wrong with today's software, perhaps that software is already dead. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 15 05:04:50 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 03:04:50 +0000 Subject: [M3devel] reference to globals in globals? Message-ID: Isn't it safe to take the address of a global? I have something like this: CONST UID_INTEGER = 1234; CONST UID_FLOAT = 4567; ... several more ... TYPE CType = OBJECT .. END; VAR t_int: CType := ...; VAR t_float: CType := ...; ... several more ... MapTypeIdToType(UID_INTEGER, t_int); MapTypeIdToType(UID_FLOAT, FLOAT); ... several more ... but what I really want is more like: TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: REF CType; END; CONST BuiltinUids = ARRAY OF BuiltinUids { BuiltinUids{UID_INTEGER, &t_int}, BuiltinUids{UID_FLOAT, &t_float}, ... several more ... }; FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); END; Heck, even if these weren't global, is it that unreasonble, from the programmer's point of view, for the language/compiler to do some pointer escape analysis and let me take the address of a local, as long as I don't store it somewhere that outlives the local? You can see this particular pattern currently in m3-sys/m3cc/gcc/gcc/m3cg/parse.c and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 where I encounter this. Working in safe languages can be frustrating... Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 15 16:51:41 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:51:41 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: Message-ID: <502BB77D.1030400@lcwb.coop> On 08/14/2012 10:04 PM, Jay K wrote: > Isn't it safe to take the address of a global? > Do you mean can't you use the ADR function in safe code if you apply it only to a global variable? The answer to that is no. The ADR function is illegal altogether in safe code. As to why, I can only speculate, but see below. I suspect even in this case, it is not as simple as it seems. > > I have something like this: > > > CONST UID_INTEGER = 1234; > CONST UID_FLOAT = 4567; > ... several more ... > > > TYPE CType = OBJECT .. END; > > > VAR t_int: CType := ...; > VAR t_float: CType := ...; > ... several more ... > > > MapTypeIdToType(UID_INTEGER, t_int); > MapTypeIdToType(UID_FLOAT, FLOAT); > ... several more ... > > > but what I really want is more like: > > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: REF CType; ^UNTRACED REF? If it were just REF, that would imply that your global variable (the pointer it contains) is a heap object, that it has heap allocator/GC overhead data attached to it, and that the GC should trace it, none of which is true. > END; > > > CONST BuiltinUids = ARRAY OF BuiltinUids { > BuiltinUids{UID_INTEGER, &t_int}, > BuiltinUids{UID_FLOAT, &t_float}, ADR instead of &? If so, you are still not there, because ADR returns a value of type ADDRESS, i.e., an untraced reference to we-don't-know-what. Somewhere, you would also have to use a LOOPHOLE to get it to UNTRACED REF CType. > ... several more ... > }; > > > FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > END; > I don't know what the signature of MapTypeIdToType is, but above, you are passing a variable of object type to its 2nd parameter, (which contains a traced reference to the actual heap object). But here, you pass the _address_ of the above. Inconsistent number of levels of indirection. A static safe language is much more likely to help with things like this. Maybe you just want to say TYPE RECORD = BuiltinUid_t = typeid: INTEGER; ctype: CType; and BuiltinUids{UID_INTEGER, t_int}? This would be equivalent to your first way, and doesn't require any unsafe coding at all. Or, you could do away with global variable t_int altogether and just initialize directly into BuiltinUids[..].ctype with whatever expression you used to initialize t_int. It looks like your array makes the t_int and cousins redundant. > > Heck, even if these weren't global, is it that unreasonble, > from the programmer's point of view, for the language/compiler > to do some pointer escape analysis and let me take the address > of a local, as long as I don't store it somewhere that outlives > the local? > This is ultimately an undecidable problem and even conservative approximations of reasonable sophistication are far too involved for a language to require of every compiler. > > You can see this particular pattern currently in > m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > > and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > where I encounter this. > > > Working in safe languages can be frustrating... > It's just an instant/deferred gratification thing. Safe languages often make you stop and fix it before you run it. Unsafe languages let you naively forge ahead to the next step, where the bug is likely to be *much* harder to diagnose, assuming you even have enough test cases to notice it at all during development. Your code here is a good example. Of course, safe languages occasionally make you unnecessarily write a bit more code to do it the safe way. E.g., the famous fake pointer to the root of a linked list example. In my personal experience, these times are at least one order of magnitude less frequent than the times safe languages reduce great pain to minor pain, albeit sooner. If fact, if you are accustomed to thinking in type-safe terms, it is seldom any harder to code it safely in the first place. You're making it too difficult. > > Thank you, > - Jay From rodney_bates at lcwb.coop Wed Aug 15 16:57:49 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 15 Aug 2012 09:57:49 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20120815043215.C8614CC85C@birch.elegosoft.com> References: <20120815043215.C8614CC85C@birch.elegosoft.com> Message-ID: <502BB8ED.7030108@lcwb.coop> On 08/15/2012 06:32 AM, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 12/08/15 06:32:15 > > Modified files: > ./: Make-lang.in > > Log message: > compile parse.c as C++ even if compiling the rest of gcc as C (or however > it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in > (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes > too much to compile as C++, plus I'm not likely to fight this battle with 4.7) Does this mean we will now have to have a C++ compiler for every target we want M3 on? > > Modified files: > cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 > > Log message: > flush some test code > > From hosking at cs.purdue.edu Wed Aug 15 18:11:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 12:11:57 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: Jay, Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. I don?t know enough about your use-case to understand what you are trying to do. On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: >> Isn't it safe to take the address of a global? >> > > Do you mean can't you use the ADR function in safe code > if you apply it only to a global variable? The answer > to that is no. The ADR function is illegal altogether in > safe code. > > As to why, I can only speculate, but see below. I suspect > even in this case, it is not as simple as it seems. > >> >> I have something like this: >> >> >> CONST UID_INTEGER = 1234; >> CONST UID_FLOAT = 4567; >> ... several more ... >> >> >> TYPE CType = OBJECT .. END; >> >> >> VAR t_int: CType := ...; >> VAR t_float: CType := ...; >> ... several more ... >> >> >> MapTypeIdToType(UID_INTEGER, t_int); >> MapTypeIdToType(UID_FLOAT, FLOAT); >> ... several more ... >> >> >> but what I really want is more like: >> >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: REF CType; > > ^UNTRACED REF? If it were just REF, that would imply that > your global variable (the pointer it contains) is a heap object, that > it has heap allocator/GC overhead data attached to it, and that the GC > should trace it, none of which is true. > > >> END; >> >> >> CONST BuiltinUids = ARRAY OF BuiltinUids { >> BuiltinUids{UID_INTEGER, &t_int}, >> BuiltinUids{UID_FLOAT, &t_float}, > > ADR instead of &? If so, you are still not there, because ADR > returns a value of type ADDRESS, i.e., an untraced reference to > we-don't-know-what. Somewhere, you would also have to use a > LOOPHOLE to get it to UNTRACED REF CType. > >> ... several more ... >> }; >> >> >> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >> END; >> > > I don't know what the signature of MapTypeIdToType is, but above, > you are passing a variable of object type to its 2nd parameter, > (which contains a traced reference to the actual heap object). > But here, you pass the _address_ of the above. Inconsistent > number of levels of indirection. A static safe language is > much more likely to help with things like this. > > Maybe you just want to say > > TYPE RECORD = BuiltinUid_t = > typeid: INTEGER; > ctype: CType; > > and > > BuiltinUids{UID_INTEGER, t_int}? > > This would be equivalent to your first way, and doesn't require any > unsafe coding at all. > > Or, you could do away with global variable t_int altogether and > just initialize directly into BuiltinUids[..].ctype with whatever > expression you used to initialize t_int. It looks like your array > makes the t_int and cousins redundant. > >> >> Heck, even if these weren't global, is it that unreasonble, >> from the programmer's point of view, for the language/compiler >> to do some pointer escape analysis and let me take the address >> of a local, as long as I don't store it somewhere that outlives >> the local? >> > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. > >> >> You can see this particular pattern currently in >> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >> >> >> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >> where I encounter this. >> >> >> Working in safe languages can be frustrating... >> > > It's just an instant/deferred gratification thing. Safe languages often > make you stop and fix it before you run it. Unsafe languages let you naively > forge ahead to the next step, where the bug is likely to be *much* harder to > diagnose, assuming you even have enough test cases to notice it at all during > development. Your code here is a good example. > > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. In my personal experience, these times are at least > one order of magnitude less frequent than the times safe languages reduce > great pain to minor pain, albeit sooner. If fact, if you are accustomed to > thinking in type-safe terms, it is seldom any harder to code it safely in the > first place. > > You're making it too difficult. > >> >> Thank you, >> - Jay > From jay.krell at cornell.edu Wed Aug 15 18:56:09 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:56:09 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: <502BB77D.1030400@lcwb.coop> Message-ID: I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable? The answer >> to that is no. The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below. I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>> ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>> ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>> ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>> typeid: INTEGER; >>> ctype: REF CType; >> >> ^UNTRACED REF? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>> BuiltinUids{UID_INTEGER, &t_int}, >>> BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what. Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>> ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above. Inconsistent >> number of levels of indirection. A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >> typeid: INTEGER; >> ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int. It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing. Safe languages often >> make you stop and fix it before you run it. Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development. Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > From jay.krell at cornell.edu Wed Aug 15 18:50:04 2012 From: jay.krell at cornell.edu (Jay) Date: Wed, 15 Aug 2012 12:50:04 -0400 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <502BB8ED.7030108@lcwb.coop> References: <20120815043215.C8614CC85C@birch.elegosoft.com> <502BB8ED.7030108@lcwb.coop> Message-ID: <34072368-7504-443C-B5EE-C0E75A75A68D@gmail.com> Yes that has already been the case for a long time. Gcc itself is also moving that way but isn't there yet. C++ compilers have long been widespread. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 10:57 AM, "Rodney M. Bates" wrote: > > > On 08/15/2012 06:32 AM, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 12/08/15 06:32:15 >> >> Modified files: >> ./: Make-lang.in >> >> Log message: >> compile parse.c as C++ even if compiling the rest of gcc as C (or however >> it is configured); this is based on what I did in gcc-apple/gcc/Makefile.in >> (gcc-apple is gcc 4.2, that Apple is staying back at, and which takes >> too much to compile as C++, plus I'm not likely to fight this battle with 4.7) > > Does this mean we will now have to have a C++ compiler for every target we want > M3 on? > >> >> Modified files: >> cm3/m3-sys/m3tests/src/p2/p251/: Main.m3 >> >> Log message: >> flush some test code >> >> > From dabenavidesd at yahoo.es Wed Aug 15 21:46:39 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 15 Aug 2012 20:46:39 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345059808.37105.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <1345059999.77923.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: we need to calculate first Module instantiation in Modula-3 in SPIN MODULE_UNIT, that is trying to establish first Module instantiation, initialization, and then you can extract CONST "address" in the actual case is similar to that, a pointer to an address of the actual value, if pointer is NIL valued then it needed initialization in compile time else is a RT error to use its value, as we have proposed in other cases. Jay, please see this BitC language, has constructors for all those RT structures of your new back-end (hope you enjoy it, see p. 14, s. 3.6.2, Tag Representation): http://bitc-lang.org/docs/bitc/spec.pdf ). This leads in some special architectures to a double pointer-sized machine word type, which is why I talked about LONGADDRESS proposal. Thanks in advance --- El mi?, 15/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] reference to globals in globals? Para: "Tony Hosking" CC: "m3devel at elegosoft.com developers" , "Jay K" Fecha: mi?rcoles, 15 de agosto, 2012 11:56 I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. I didn't want the array only because then I could only access the data more verbosely/slowly via the array.? More later, maybe. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > Jay, > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > I don?t know enough about your use-case to understand what you are trying to do. > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> Isn't it safe to take the address of a global? >>> >> >> Do you mean can't you use the ADR function in safe code >> if you apply it only to a global variable?? The answer >> to that is no.? The ADR function is illegal altogether in >> safe code. >> >> As to why, I can only speculate, but see below.? I suspect >> even in this case, it is not as simple as it seems. >> >>> >>> I have something like this: >>> >>> >>> CONST UID_INTEGER = 1234; >>> CONST UID_FLOAT = 4567; >>>? ... several more ... >>> >>> >>> TYPE CType = OBJECT .. END; >>> >>> >>> VAR t_int: CType := ...; >>> VAR t_float: CType := ...; >>>? ... several more ... >>> >>> >>> MapTypeIdToType(UID_INTEGER, t_int); >>> MapTypeIdToType(UID_FLOAT, FLOAT); >>>? ... several more ... >>> >>> >>> but what I really want is more like: >>> >>> >>> TYPE RECORD = BuiltinUid_t = >>>? typeid: INTEGER; >>>? ctype: REF CType; >> >>? ? ? ? ???^UNTRACED REF?? If it were just REF, that would imply that >> your global variable (the pointer it contains) is a heap object, that >> it has heap allocator/GC overhead data attached to it, and that the GC >> should trace it, none of which is true. >> >> >>> END; >>> >>> >>> CONST BuiltinUids = ARRAY OF BuiltinUids { >>>? BuiltinUids{UID_INTEGER, &t_int}, >>>? BuiltinUids{UID_FLOAT, &t_float}, >> >> ADR instead of &?? If so, you are still not there, because ADR >> returns a value of type ADDRESS, i.e., an untraced reference to >> we-don't-know-what.? Somewhere, you would also have to use a >> LOOPHOLE to get it to UNTRACED REF CType. >> >>>? ... several more ... >>> }; >>> >>> >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO >>>? MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); >>> END; >>> >> >> I don't know what the signature of MapTypeIdToType is, but above, >> you are passing a variable of object type to its 2nd parameter, >> (which contains a traced reference to the actual heap object). >> But here, you pass the _address_ of the above.? Inconsistent >> number of levels of indirection.? A static safe language is >> much more likely to help with things like this. >> >> Maybe you just want to say >> >> TYPE RECORD = BuiltinUid_t = >>? typeid: INTEGER; >>? ctype: CType; >> >> and >> >> BuiltinUids{UID_INTEGER, t_int}? >> >> This would be equivalent to your first way, and doesn't require any >> unsafe coding at all. >> >> Or, you could do away with global variable t_int altogether and >> just initialize directly into BuiltinUids[..].ctype with whatever >> expression you used to initialize t_int.? It looks like your array >> makes the t_int and cousins redundant. >> >>> >>> Heck, even if these weren't global, is it that unreasonble, >>> from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. >> >>> >>> You can see this particular pattern currently in >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c >>> >>> >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 >>> where I encounter this. >>> >>> >>> Working in safe languages can be frustrating... >>> >> >> It's just an instant/deferred gratification thing.? Safe languages often >> make you stop and fix it before you run it.? Unsafe languages let you naively >> forge ahead to the next step, where the bug is likely to be *much* harder to >> diagnose, assuming you even have enough test cases to notice it at all during >> development.? Your code here is a good example. >> >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way.? E.g., the famous fake pointer to the root of a >> linked list example.? In my personal experience, these times are at least >> one order of magnitude less frequent than the times safe languages reduce >> great pain to minor pain, albeit sooner.? If fact, if you are accustomed to >> thinking in type-safe terms, it is seldom any harder to code it safely in the >> first place. >> >> You're making it too difficult. >> >>> >>> Thank you, >>> - Jay >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:07:20 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:07:20 +0000 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: > If fact, if you are accustomed to thinking in type-safe terms I do think in fairly type-terms. In C++ I know to use templates, for example. There is a matter of lifetime-safety though. I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. Consider C++ code like this, where I want to do some things "the same" to a bunch of variables, and some things "specific". struct Person_t { const char* name; }; void F() { Person_t jay {"jay"}; Person_t randy = {"randy"}; Person_t tony = {"tony"}; // print all their names Person_t* people[] = {&jay, &randy, &tony}; for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i]->name); DoSomethingSpecificToJay(&jay); DoSomethingSpecificToRandy(&randy); DoSomethingSpecificToTony(&tony); } I do NOT want to do this: void F() { Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; // print all their names for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) printf("%s\n", people[i].name); DoSomethingSpecificToJay(&people[0]); DoSomethingSpecificToRandy(&people[1]); DoSomethingSpecificToTony(&people[2]); } I understand there is: enum People { jay, randy, tony }; ... DoSomethingSpecificToJay(&people[jay]); DoSomethingSpecificToRandy(&people[randy]); DoSomethingSpecificToTony(&people[tony]); ... but I'm still not thrilled about that. I ended up without anything "specific" except initialization..so no really need to hang a name/variable on the data, and the initialization is fairly specific, so the resulting code is merely: EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); ... you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems in the m3cg interface...these types need to be defined via function calls into the backend, not hardcoded..and even if they are...the order of types arriving at the backend isn't always ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) - Jay > From: jay.krell at cornell.edu > Date: Wed, 15 Aug 2012 12:56:09 -0400 > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] reference to globals in globals? > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > Jay, > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/14/2012 10:04 PM, Jay K wrote: > >>> Isn't it safe to take the address of a global? > >>> > >> > >> Do you mean can't you use the ADR function in safe code > >> if you apply it only to a global variable? The answer > >> to that is no. The ADR function is illegal altogether in > >> safe code. > >> > >> As to why, I can only speculate, but see below. I suspect > >> even in this case, it is not as simple as it seems. > >> > >>> > >>> I have something like this: > >>> > >>> > >>> CONST UID_INTEGER = 1234; > >>> CONST UID_FLOAT = 4567; > >>> ... several more ... > >>> > >>> > >>> TYPE CType = OBJECT .. END; > >>> > >>> > >>> VAR t_int: CType := ...; > >>> VAR t_float: CType := ...; > >>> ... several more ... > >>> > >>> > >>> MapTypeIdToType(UID_INTEGER, t_int); > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > >>> ... several more ... > >>> > >>> > >>> but what I really want is more like: > >>> > >>> > >>> TYPE RECORD = BuiltinUid_t = > >>> typeid: INTEGER; > >>> ctype: REF CType; > >> > >> ^UNTRACED REF? If it were just REF, that would imply that > >> your global variable (the pointer it contains) is a heap object, that > >> it has heap allocator/GC overhead data attached to it, and that the GC > >> should trace it, none of which is true. > >> > >> > >>> END; > >>> > >>> > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > >>> BuiltinUids{UID_INTEGER, &t_int}, > >>> BuiltinUids{UID_FLOAT, &t_float}, > >> > >> ADR instead of &? If so, you are still not there, because ADR > >> returns a value of type ADDRESS, i.e., an untraced reference to > >> we-don't-know-what. Somewhere, you would also have to use a > >> LOOPHOLE to get it to UNTRACED REF CType. > >> > >>> ... several more ... > >>> }; > >>> > >>> > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > >>> END; > >>> > >> > >> I don't know what the signature of MapTypeIdToType is, but above, > >> you are passing a variable of object type to its 2nd parameter, > >> (which contains a traced reference to the actual heap object). > >> But here, you pass the _address_ of the above. Inconsistent > >> number of levels of indirection. A static safe language is > >> much more likely to help with things like this. > >> > >> Maybe you just want to say > >> > >> TYPE RECORD = BuiltinUid_t = > >> typeid: INTEGER; > >> ctype: CType; > >> > >> and > >> > >> BuiltinUids{UID_INTEGER, t_int}? > >> > >> This would be equivalent to your first way, and doesn't require any > >> unsafe coding at all. > >> > >> Or, you could do away with global variable t_int altogether and > >> just initialize directly into BuiltinUids[..].ctype with whatever > >> expression you used to initialize t_int. It looks like your array > >> makes the t_int and cousins redundant. > >> > >>> > >>> Heck, even if these weren't global, is it that unreasonble, > >>> from the programmer's point of view, for the language/compiler > >>> to do some pointer escape analysis and let me take the address > >>> of a local, as long as I don't store it somewhere that outlives > >>> the local? > >>> > >> > >> This is ultimately an undecidable problem and even conservative > >> approximations of reasonable sophistication are far too involved > >> for a language to require of every compiler. > >> > >>> > >>> You can see this particular pattern currently in > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > >>> > >>> > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > >>> where I encounter this. > >>> > >>> > >>> Working in safe languages can be frustrating... > >>> > >> > >> It's just an instant/deferred gratification thing. Safe languages often > >> make you stop and fix it before you run it. Unsafe languages let you naively > >> forge ahead to the next step, where the bug is likely to be *much* harder to > >> diagnose, assuming you even have enough test cases to notice it at all during > >> development. Your code here is a good example. > >> > >> Of course, safe languages occasionally make you unnecessarily write a bit more > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > >> linked list example. In my personal experience, these times are at least > >> one order of magnitude less frequent than the times safe languages reduce > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > >> first place. > >> > >> You're making it too difficult. > >> > >>> > >>> Thank you, > >>> - Jay > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:11:25 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:11:25 +0000 Subject: [M3devel] m3 backends and builtin types? Message-ID: Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 00:15:51 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Aug 2012 22:15:51 +0000 Subject: [M3devel] some proposed m3cg additions? Message-ID: I suggest backends should be informed of the following somehow via m3cg calls: word size definitely endian maybe jmpbuf size maybe target_name maybe I understand they are available from m3middle. Part of my agenda is that .mc files be more self contained. Eventually I'd like the IL to be independent of all of these anyway, but that is a later project. Jmpbuf size I know how to eliminate. Target_name doesn't really have meaning. Eliminating word size and endian dependencies in the IL won't be so easy. The extra information should be pretty harmless even if not critical. For example though...if you look at m3cgcat..it is slightly bad that it has to discover target from the ambient environment, instead of from the input file. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 01:58:23 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 15 Aug 2012 19:58:23 -0400 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: Message-ID: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote: > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so, RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 16 02:55:12 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 15 Aug 2012 20:55:12 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502BB77D.1030400@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> Message-ID: <20120816005512.GA31309@topoi.pooq.com> On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures? the ones whose environments are completely static? Or am I thinking of another type-safe language? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way. E.g., the famous fake pointer to the root of a > linked list example. You mean something like (in C with probable errors) list **p, *q; make q point to some linked list for ( p = &q; *p != NIL; p = &((*p)->next)) { And here you can easily delete elements by *p = (*p) -> next } This can be done prefectly type-safe in languages that allow REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik From dabenavidesd at yahoo.es Thu Aug 16 03:14:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 02:14:27 +0100 (BST) Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> Message-ID: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html Thanks in advance --- El mi?, 15/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] reference to globals in globals? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 15 de agosto, 2012 19:55 On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > >Heck, even if these weren't global, is it that unreasonble, > >from the programmer's point of view, for the language/compiler > >to do some pointer escape analysis and let me take the address > >of a local, as long as I don't store it somewhere that outlives > >the local? > > > > This is ultimately an undecidable problem and even conservative > approximations of reasonable sophistication are far too involved > for a language to require of every compiler. Not to mention the obscurityof the language definition. But isn't there something like that with the addresses of top-level proocedures?? the ones whose environments are completely static? Or am I thinking of another type-safe language?? Or am I thinking of assigning the procedures themselves? -- hendrik > Of course, safe languages occasionally make you unnecessarily write a bit more > code to do it the safe way.? E.g., the famous fake pointer to the root of a > linked list example.? You mean something like (in C with probable errors) ? list **p, *q; ? make q point to some linked list ? for ( p = &q; *p != NIL; p = &((*p)->next)) ? { ???And here you can easily delete elements by *p = (*p) -> next ? } This can be done prefectly type-safe in languages that allow ???REF REF list Or is there some other fake pointer trick you're talking about? -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 04:07:52 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 02:07:52 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: > There is a bootstrap issue here isn?t there? I am not sure. I don't understand the system enough. :( I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. The compiler does for example construct the type BOOLEAN out of lower level pieces. Can it not then proceed to compute its typeid, and inform the backend about it? Other examples appear to include...wait.where did I get this list? At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). I built up the following list: UID_INTEGER = 16_195C2A74; (* INTEGER *) UID_LONGINT = 16_05562176; (* LONGINT *) UID_WORD = 16_97E237E2; (* CARDINAL *) UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) UID_REEL = 16_48E16572; (* REAL *) UID_LREEL = 16_94FE32F6; (* LONGREAL *) UID_XREEL = 16_9EE024E3; (* EXTENDED *) UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) UID_CHAR = 16_56E16863; (* CHAR [0..255] *) UID_WIDECHAR = 16_88F439FC; UID_MUTEX = 16_1541F475; (* MUTEX *) UID_TEXT = 16_50F86574; (* TEXT *) UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) UID_ROOT = 16_9D8FB489; (* ROOT *) UID_REFANY = 16_1C1C45E6; (* REFANY *) UID_ADDR = 16_08402063; (* ADDRESS *) UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) UID_NULL = 16_48EC756E; (* NULL *) Surely these aren't all fundamental?? Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: TYPE Record1 = RECORD REF Record2 END; TYPE Record2 = RECORD REF Record1 END; If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. - Jay From: hosking at cs.purdue.edu Date: Wed, 15 Aug 2012 19:58:23 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3 backends and builtin types? There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... On Aug 15, 2012, at 6:11 PM, Jay K wrote:Something seems off to me in the current implementation. Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. I hardcoded knowledge of them in parse.c and M3C.m3. That seems wrong. Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. Shouldn't m3front inform the backend via m3cg of these types? It is doable using the existing interfaces? More so, RTBuiltin.mx ought not exist, right? Whatever data it contains should be built up like any other type data? Part of the same problem? - Jay Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 06:23:37 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 04:23:37 +0000 Subject: [M3devel] m3middle/TFloat Message-ID: This TFloat module doesn't look right to me. I guess, ok, in a sense -- if host floating point is adequate.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 15:58:59 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 13:58:59 +0000 Subject: [M3devel] m3cg.set_target_name, set_endian, set_wordsize? Message-ID: I already said this, but more concretely, I propose M3CG_Ops additions: set_target_name(name: TEXT); TYPE Endian = { Little, Big }; (* or Endian_t or Endian_e *) set_endian(endian: Endian); set_wordsize(wordsize: INTEGER); (* currently always 32 or 64 *) I MIGHT have use of these in m3back/M3C. I'm not sure yet. target_name implies endian and wordsize and more, but it a kind of vague thing. It also implies win32 vs. Posix, which M3C probably won't need. It also implies jmpbuf size, which might be useful, or maybe we remove that "the other way". You know -- I want to eventually remove target knowledge from m3front, and have none in M3C either. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 16 16:21:01 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 16 Aug 2012 14:21:01 +0000 Subject: [M3devel] higher level m3cg? Message-ID: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Aug 16 17:06:18 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Thu, 16 Aug 2012 11:06:18 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Yes, I've thought so for a long time. It would also more easily enable llvm. Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 17:53:58 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 16:53:58 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345132438.30720.YahooMailClassic@web29701.mail.ird.yahoo.com> HI all: OO C is C++ in theory not in practice, could be better than this, but we should not do so and don't look a Modula-3 type system that preserves under compilation that's an abstract machine that interacts with whatever machine code generator is (M3CG) compiled to for whatever language you would like? That's why I insist the original acronym of M3CG(M2CG) is Machine Code Generator, this is not a translator, nor anything else, it was created for the purpose of compiling and linking compiler intermediate language, nothing else. Besides many languages and tongues I guess is better to speak clearly in a universal machine like Baby Modula-3, choosing Modula-3 as mother tongue, that is we need to speak in Modula-3 from Baby and counter wise. As some TV add said: "the problem is simple the answer it's not" So I would try to explain meaningfully clearly, we need to understand how to encode Modula-3 (1989) in a language like Quest (1994) in terms of Baby Modula-3 (1993), that is Target is BM3, Host is Quest, and Source is Modula-3. It turns out that doing type inference in terms of Quest is decidable, but in terms of Baby Modula-3 I don't know, it's pretty much a question of the million, the problem for real is can we encode Modula-3 in Baby Modula-3? That's a good question for the people who advocate for not so obscure language definition (It has some examples of understanding the language in terms of a exception key concept, but not in object terms) So, what is next is to find a theory of Objects, there are people that is not happy with Cardelli's definition and I agree with them we should we looking at the problem concretely with options I know of different kinds of options, but you would not like to hear the one I prefer is likely to be undecidable and inconsistent terms, so we could argue about obscurity of language definition again (right ?). This option is called pebble is a language written by Butler Lampson (Modula-2+ author) and Rodney Burstall (a aforementioned computer theorist ). In some sense Pebble is just about to talk about Modula-3 Module system, but it would need a lot of research to put it on place for being chosen. Thanks in advance --- El jue, 16/8/12, Jay K escribi?: De: Jay K Asunto: [M3devel] higher level m3cg? Para: "m3devel" Fecha: jueves, 16 de agosto, 2012 09:21 Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 16 18:40:45 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 11:40:45 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> References: <1345079667.62906.YahooMailClassic@web29704.mail.ird.yahoo.com> Message-ID: <502D228D.7070306@lcwb.coop> On 08/15/2012 08:14 PM, Daniel Alejandro Benavides D. wrote: > Hi all: > For sure, do you mean Top in type sense? RT0 is the bottom of the world, but type here are "bottom of the world" > http://modula3.elegosoft.com/cm3/doc/help/gen_html/m3core/src/runtime/common/RT0.i3.html > > Thanks in advance > I think here, "top" means outermost in the nested scopes of a compilation. That is, a "top-level" procedure is declared immediately inside an interface or module, not inside any procedure or block. This is an entirely different hierarchy from that of interfaces importing other interfaces, where the "bottom of the world" phrase is used to mean these types do not refer to any other imported interface. Oddly, I have heard this second sense referred to as the "top" of a type hierarchy, but that usage is highly counterintuitive to me. > --- El *mi?, 15/8/12, Hendrik Boom //* escribi?: > > > De: Hendrik Boom > Asunto: Re: [M3devel] reference to globals in globals? > Para: m3devel at elegosoft.com > Fecha: mi?rcoles, 15 de agosto, 2012 19:55 > > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > > > > > > On 08/14/2012 10:04 PM, Jay K wrote: > > > > > >Heck, even if these weren't global, is it that unreasonble, > > >from the programmer's point of view, for the language/compiler > > >to do some pointer escape analysis and let me take the address > > >of a local, as long as I don't store it somewhere that outlives > > >the local? > > > > > > > This is ultimately an undecidable problem and even conservative > > approximations of reasonable sophistication are far too involved > > for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > > > Of course, safe languages occasionally make you unnecessarily write a bit more > > code to do it the safe way. E.g., the famous fake pointer to the root of a > > linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } > > This can be done prefectly type-safe in languages that allow > > REF REF list > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Thu Aug 16 19:27:17 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 12:27:17 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502D2D75.8010702@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > Yes, yes, yes!!! > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? Yes. If you can assign a procedure (or pointer to function, which is equivalent, for this discussion) to a variable, then there are ways to call the procedure when its environment does not exist, i.e, one or more scopes of variables that the procedure is declared inside of and can refer to. The outermost scope always exists, so we need three nested scopes to get an example. MODULE Outer ; PROCEDURE Middle ( MiddleFormal : INTEGER ) = PROCEDURE Inner ( ) : INTEGER = BEGIN RETURN MiddleFormal END ; BEGIN (* Middle *) ProcVar := Inner (* Statically illegal because Inner is not top-level. *) END (* Middle ; VAR ProcVar : PROCEDURE ( ) : INTEGER ; VAR X : INTEGER ; BEGIN (* Outer *) Middle ( 7 ) ; X := ProcVar ( ) END Outer . If the assignment ProcVar := Inner were legal, the call ProcVar ( ) goes to Inner, which refers to MiddleFormal of Middle, but there is no active invocation of Middle, and MiddleFormal doesn't exist. Pascal (if I remember all this right) allows procedures to be passed as parameters, but not assigned to variables. The procedure constant in question can only be named where it has an environment. If all you can do with it is pass it, any use of it can only be deeper inside the dynamic call chain, where its environment continues to exist. Modula-2 allows procedures to be either passed or assigned, but in either case, only a top-level procedure, which always has an environment, can be given. Both passing and assigning procedures have reasonable uses, both for variations on the callback theme. Modula-3 supports both by allowing any procedure to be passed but only a top-level one to be assigned. Enforcing the latter restriction sometimes means a runtime check at the point of an assignment, if the RHS of the assignment is a procedure formal parameter, which could dynamically be top-level or not. This is probably not a high-frequency situation. Assigning a procedure constant and passing any procedure can be fully checked at compile time. I think it is possible to define a weaker, yet safe rule that postpones runtime environment existence checking until the point of a call on a procedure variable, but I don't recall seeing this in any language. Maybe Algol68 or something. In gcc-enhanced C, which allows nested functions, in addition to both assigning and passing (pointers to) functions, you are on your own to avoid calling something that lacks an environment. The one help the language gives is to say in the reference, "all hell will break loose" if you get this wrong. > > -- hendrik From rodney_bates at lcwb.coop Thu Aug 16 20:16:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 16 Aug 2012 13:16:35 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502D3903.6020606@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: This is from memory, is incomplete, and my not be 100% reliable, but hopefully useful. There are two different hash codes on type structures used in (at least) the compiler, runtime, and pickles. The "uid" is 32-bit, and the "signature" is 64-bit. I think the signature maybe only applies to reference types. It is what is used in pickles. One of them takes into account some properties the other does not. I am not entirely sure which of the following information applies to which hash code(s). At one time, I think I had found a main program that computed one or the other of these, or both. They are not arbitrary, they are somehow derived from the actual type structures. (What are the type structures for primitive builtin types?) For builtin types, they have apparently been hand copied to several places. (compiler, RTS, pickles, m3gdb, ...?) They also get computed for programmer-defined types too, but obviously you won't find these hard-coded. However, some of the values for builtin types have suffered some alterations. Here is an excerpt from m3gdb, m3-lang.c, with a possibly relevant comment that I once inferred somehow: if (m3uid_to_num (uid, &num_uid)) { if (num_uid == 0x195c2a74) { return builtin_type_m3_integer; } else if (num_uid == 0x05562176) { return builtin_type_m3_longint; } else if (num_uid == 0x50f86574) { return builtin_type_m3_text; } else if (num_uid == 0x97e237e2) { return builtin_type_m3_cardinal; } else if (num_uid == 0x1e59237d) { return builtin_type_m3_boolean; } else if (num_uid == 0x08402063) { return builtin_type_m3_address; } else if (num_uid == 0x9d8fb489) { return builtin_type_m3_root; } else if (num_uid == 0xa973d3a6) { return builtin_type_m3_transient_root; } else if (num_uid == 0x56e16863) { return builtin_type_m3_char; } /* For widechar, the num_uid was once 0xb0830411. Presumably, this is an outdated leftover from a transitional implementation of widechar, at Critical Mass, before the final one came from them. */ else if (num_uid == 0x88f439fc) { return builtin_type_m3_widechar; } else if (num_uid == 0x48e16572) { return builtin_type_m3_real; } else if (num_uid == 0x94fe32f6) { return builtin_type_m3_longreal; } else if (num_uid == 0x9ee024e3) { return builtin_type_m3_extended; } else if (num_uid == 0x48ec756e) { return builtin_type_m3_null; } else if (num_uid == 0x1c1c45e6) { return builtin_type_m3_refany; } else if (num_uid == 0x51e4b739) { return builtin_type_m3_transient_refany; } else if (num_uid == 0x898ea789) { return builtin_type_m3_untraced_root; } else if (num_uid == 0x00000000) { return builtin_type_m3_void; } } Also, there is some endian-related confusion, at least for signatures. Sometimes they are stored in an array of bytes, whose order may be always the same, regardless of host or target. However, some were apparently hand-reordered between pm3 and cm3, and the reorderings are neither strictly endianness nor consistent for different types. I seem to recall something like swapping 32-bit segments, but not bytes within them, or the other way around, or something, for some but not all types. I had to deal with both versions of the orderings in pickles and maybe in m3gdb. Here is an excerpt from Pickle2.m3: (* Adaptation to some builtin fingerprints that have different byte order in Pm3 and Cm3. *) TYPE FPA = ARRAY [0..7] OF BITS 8 FOR [0..255]; (* Give a short name to anonymous type of Fingerprint.T.byte. *) CONST NULL_uid = 16_48ec756e; CONST pm3_NULL_Fp = FPA {16_24,16_80,16_00,16_00,16_6c,16_6c,16_75,16_6e}; CONST cm3_NULL_Fp = FPA {16_6e,16_75,16_6c,16_6c,16_00,16_00,16_80,16_24}; CONST ROOT_uid = 16_9d8fb489; CONST pm3_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_65,16_86,16_ad,16_41}; CONST cm3_ROOT_Fp = FPA {16_41,16_ad,16_86,16_65,16_c8,16_19,16_09,16_f8}; CONST UNTRACED_ROOT_uid = 16_898ea789; CONST pm3_UNTRACED_ROOT_Fp = FPA {16_f8,16_09,16_19,16_c8,16_71,16_87,16_be,16_41}; CONST cm3_UNTRACED_ROOT_Fp = FPA {16_41,16_be,16_87,16_71,16_c8,16_19,16_09,16_f8}; (* Can the following two occur in a pickle? Maybe if somebody registered a special for them? *) CONST ADDRESS_uid = 16_08402063; CONST pm3_ADDRESS_Fp = FPA {16_91,16_21,16_8a,16_62,16_f2,16_01,16_ca,16_6a}; CONST cm3_ADDRESS_Fp = FPA {16_f2,16_01,16_ca,16_6a,16_91,16_21,16_8a,16_62}; CONST REFANY_uid = 16_1c1c45e6; CONST pm3_REFANY_Fp = FPA {16_65,16_72,16_24,16_80,16_79,16_6e,16_61,16_66}; CONST cm3_REFANY_Fp = FPA {16_66,16_61,16_6e,16_79,16_80,16_24,16_72,16_65}; Hope this is a step forward in the way of helpful information > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From jay.krell at cornell.edu Thu Aug 16 20:43:02 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 16 Aug 2012 14:43:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> References: <0D27AB94-877E-46DF-BBAD-09292DFCAF85@cs.purdue.edu> Message-ID: Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised". I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient. But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis. I have to figure out something for nested functions... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: > Yes, I've thought so for a long time. It would also more easily enable llvm. > > Sent from my iPhone > > On Aug 16, 2012, at 10:21, Jay K wrote: > >> Should m3cg provide enough information for a backend to generate idiomatic C? >> (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) >> >> >> Should we make it so? >> >> >> Or be pragmatic and see if anyone gets to that point? >> >> >> But, look at this another way. >> Let's say we are keeping the gcc backend. >> >> >> Isn't it reasonable to have a better experience with stock gdb? >> >> >> What should m3cg look like then? >> >> >> Matching up m3front to gcc turns out to be "wierd". >> As does having a backend generate "C". >> >> >> In particular, "wierd" because there is a "level mismatch". >> >> >> m3cg presents a fairly low level view of the program. >> It does layout. Global variables are stuffed into what you might call a "struct", with >> no assigned field names. Field references are done by adding to addresses and casting. >> >> >> Too low level to provide a "good" gcc tree representation or to generate "normal" C. >> >> >> One might be able to, by somewhat extraordinary means, make due. >> That is, specifically one could deduce field references from >> offsets/sizes. But maybe it is reasonable for load/store >> to include fields? Maybe in addition to what it provides? >> >> >> As well, it appears to me, that >> >> >> given TYPE Enum = {One, Two, Three}; >> >> the m3cg is like: >> >> declare enum typeidblah >> declare enum_elt One >> declare enum_elt Two >> declare enum_elt Three >> declare_typename typeidblah Enum >> >> >> One kind of instead wants more like: >> >> >> declare enum typeidblah Enum >> declare enum_elt One => rename it Enum_One >> declare enum_elt Two "" >> declare enum_elt Three "" >> >> >> However I understand that {One, Two, Three} exists >> as anonymous type independent of the name "Enum". >> >> >> One could just as well have: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = {One, Two, Three}; >> >> >> Enum1 and Enum2 probably have the same typeid, and are just >> two typenames for the same type. >> >> >> likewise: >> given TYPE Enum1 = {One, Two, Three}; >> given TYPE Enum2 = Enum1; >> >> >> but, pragmatically, in the interest of generating better C, >> can we pass a name along with declare_enum? >> >> I ask somewhat rhetorically. I realize there is the answer: >> enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >> typedef enum Mtypeid Enum1; >> >> >> Also, enum variables I believe end up as just UINT8, 16, or 32. >> Loads of enum values I believe end up as just loads of integers. >> Can we pass along optional enum names with declare_local/declare_param? >> And optional enum names with load_int? >> Or add a separate load_enum call? >> >> >> Really, I understand that the current interface can be pressed to do >> pretty adequate things. I can infer field references. The way enums work >> isn't too bad. >> >> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 16 23:13:55 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 16 Aug 2012 22:13:55 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345151635.43876.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: Jay, even in RISC like G. Nelson mentioned there was this problem of big executables, and I happen to believe this doesn't change things too much for the better, so be careful with summing all kind of stuff will go on OK, there could be the side effects that we don't like too much. In the other side SPIN M3 Vortex compiler was a DEC-SRC modified front-end for that back-end to a Java-like subset backend, I don't know how well it was that, but at least they have done that already to the front-end so is less work to do. I don't care if you redefine the language to admittedly drop some unwanted changes but with some purpose of simplifying things, this was the best effort they did so we need to be aware of that. I generally don't take to heart this part of the system, but we need to respect the existence of this backend if somebody has a new better and brave CG that's better but to use your own language to do this I don't like the idea of several backends and different languages. Thanks in advance --- El jue, 16/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Antony Hosking" CC: "m3devel" , "Jay K" Fecha: jueves, 16 de agosto, 2012 13:43 Btw, while I claimed to ignore them...a higher level information about exception handling to the backend might be good. & possibly loops. It really just depends on what the backend is outputting & how much work it is willing to do. There is kind of this "gradual lowering" going on..but sometimes things get "reraised".? I guess really there is no canonical set in stone interface & implementation. We'll figure it out gradually. But I also am not as familiar with the system as I would like. On the matter of exception handling, it 'd likely be good to generate __try for NT platforms and C++ try otherwise. It'd likely make things more efficient.?But that would snowball a bit -- generating the catch/__except code would likely require other higher level information. On the matter of LLVM I don't expect to have the time/motivation. Anyway, I'll press on for asis.I have to figure out something for nested functions... ?- Jay (briefly/pocket-sized-computer-aka-phone) On Aug 16, 2012, at 11:06 AM, Antony Hosking wrote: Yes, I've thought so for a long time. It would also more easily enable llvm.? Sent from my iPhone On Aug 16, 2012, at 10:21, Jay K wrote: Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 17 17:16:21 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 17 Aug 2012 11:16:21 -0400 Subject: [M3devel] reference to globals in globals? In-Reply-To: <502D2D75.8010702@lcwb.coop> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> <502D2D75.8010702@lcwb.coop> Message-ID: <20120817151621.GB19487@topoi.pooq.com> On Thu, Aug 16, 2012 at 12:27:17PM -0500, Rodney M. Bates wrote: > > > On 08/15/2012 07:55 PM, Hendrik Boom wrote: > >On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: > >> > >> > >>On 08/14/2012 10:04 PM, Jay K wrote: > >>> > >>>Heck, even if these weren't global, is it that unreasonble, > >>>from the programmer's point of view, for the language/compiler > >>>to do some pointer escape analysis and let me take the address > >>>of a local, as long as I don't store it somewhere that outlives > >>>the local? > >>> > >> > >>This is ultimately an undecidable problem and even conservative > >>approximations of reasonable sophistication are far too involved > >>for a language to require of every compiler. > > > >Not to mention the obscurityof the language definition. > > > > Yes, yes, yes!!! > > >But isn't there something like that with the addresses of top-level > >proocedures? the ones whose environments are completely static? > >Or am I thinking of another type-safe language? Or am I thinking of > >assigning the procedures themselves? > > Yes. If you can assign a procedure (or pointer to function, which is > equivalent, for this discussion) to a variable, then there are ways to > call the procedure when its environment does not exist, i.e, one or > more scopes of variables that the procedure is declared inside of and > can refer to. The outermost scope always exists, so we need three > nested scopes to get an example. > > MODULE Outer > > ; PROCEDURE Middle ( MiddleFormal : INTEGER ) > > = PROCEDURE Inner ( ) : INTEGER > > = BEGIN > RETURN MiddleFormal > END > > ; BEGIN (* Middle *) > ProcVar := Inner (* Statically illegal because Inner is not top-level. *) > END (* Middle > > ; VAR ProcVar : PROCEDURE ( ) : INTEGER > ; VAR X : INTEGER > > ; BEGIN (* Outer *) > Middle ( 7 ) > ; X := ProcVar ( ) > END Outer > . > > If the assignment ProcVar := Inner were legal, the call ProcVar ( ) > goes to Inner, which refers to MiddleFormal of Middle, but there is no > active invocation of Middle, and MiddleFormal doesn't exist. > > Pascal (if I remember all this right) allows procedures to be passed > as parameters, but not assigned to variables. The procedure constant > in question can only be named where it has an environment. If all you > can do with it is pass it, any use of it can only be deeper inside the > dynamic call chain, where its environment continues to exist. > > Modula-2 allows procedures to be either passed or assigned, but in > either case, only a top-level procedure, which always has an > environment, can be given. > > Both passing and assigning procedures have reasonable uses, both for > variations on the callback theme. > > Modula-3 supports both by allowing any procedure to be passed but only > a top-level one to be assigned. Enforcing the latter restriction > sometimes means a runtime check at the point of an assignment, if the > RHS of the assignment is a procedure formal parameter, which could > dynamically be top-level or not. This is probably not a > high-frequency situation. Assigning a procedure constant and passing > any procedure can be fully checked at compile time. > > I think it is possible to define a weaker, yet safe rule that > postpones runtime environment existence checking until the point of a > call on a procedure variable, but I don't recall seeing this in any > language. Maybe Algol68 or something. Algol 68 restricted assignment. The 'scope' of the procedure had to be at most as deeply nested as the 'scope' of the variable. This was a general restriction that applied to everything, though, not just procedures. This meant you could assign nested procedures to more nested variables. The 'scope' of a procedure was determined by its most local global variable. But scope checking led to heavy overhead on just about anything having to do with pointers, so it tended to be rarely inplemented. As on implementer said, this isn't the kind of mistake programmers make, anyway. The CDC Cyber implementation performed the check when called. Each scope (or rather, each scope for with this situation could arise) caused a small object to be allocated on the heap on scope entry. The stack frame would point to this object, and the object would point back to the stack frame. A procedure would contain a pointer to this object for the stak frame contining its most local globals.. WHen the procedure was called, it would check that this object indeed pointed to the place on the stack frame where there would be a pointer back to the object. Still, the restriction eliminated the possibility of writing a function-composition function. The only way I see of making this more liberal is to perform escape analysis on procedures, and subsequently choosing a few blocks whose stack frame would be allocated on the heap. You could apply run-time tests to prevent you from jumping into stack frames that have already been exited (of course in Modula 3, there aren't any GOTOs. It turns out that the analysis isn't all that difficult, and that for normal programs, almost no frames get allocated on the heap, hence little run-time overhead. The mechanism could be useful for callbacks. This is is the kind of analysis every decent Scheme compiler does to avoid accomodating the full force of conitinuation passing where it is not needed. -- hendrik > > In gcc-enhanced C, which allows nested functions, in addition to both > assigning and passing (pointers to) functions, you are on your own to > avoid calling something that lacks an environment. The one help the > language gives is to say in the reference, "all hell will break loose" > if you get this wrong. Does gcc-enhanced C allow you to put stack frames on the heap? -- hendrik From rodney_bates at lcwb.coop Fri Aug 17 20:51:12 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 13:51:12 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: <20120816005512.GA31309@topoi.pooq.com> References: <502BB77D.1030400@lcwb.coop> <20120816005512.GA31309@topoi.pooq.com> Message-ID: <502E92A0.5080809@lcwb.coop> On 08/15/2012 07:55 PM, Hendrik Boom wrote: > On Wed, Aug 15, 2012 at 09:51:41AM -0500, Rodney M. Bates wrote: >> >> >> On 08/14/2012 10:04 PM, Jay K wrote: >>> >>> Heck, even if these weren't global, is it that unreasonble, >> >from the programmer's point of view, for the language/compiler >>> to do some pointer escape analysis and let me take the address >>> of a local, as long as I don't store it somewhere that outlives >>> the local? >>> >> >> This is ultimately an undecidable problem and even conservative >> approximations of reasonable sophistication are far too involved >> for a language to require of every compiler. > > Not to mention the obscurityof the language definition. > > But isn't there something like that with the addresses of top-level > proocedures? the ones whose environments are completely static? > Or am I thinking of another type-safe language? Or am I thinking of > assigning the procedures themselves? > > -- hendrik > >> Of course, safe languages occasionally make you unnecessarily write a bit more >> code to do it the safe way. E.g., the famous fake pointer to the root of a >> linked list example. > > You mean something like (in C with probable errors) > > list **p, *q; > make q point to some linked list > > for ( p = &q; *p != NIL; p = &((*p)->next)) > { > And here you can easily delete elements by *p = (*p) -> next > } Hmm, this is not quite what I had in mind, but it's close. I like to code things like this with a pointer to the interesting node _and_ one that lags one node behind, in case list surgery is needed. Also, a more complete example would refer to one or more data fields of a node too. But I think I can illustrate my intentions with your example. > > This can be done prefectly type-safe in languages that allow > > REF REF list Plus, an ADR/& operation. In the case of C's &, it's type at least remembers the type of the object whose address is taken. Modula-3's ADR does not, so would require a LOOPHOLE as well. But no matter, because an operator that can take the address of anything other than a heap object (or maybe a component of one?) is unsafe. > > Or is there some other fake pointer trick you're talking about? > > -- hendrik > From rodney_bates at lcwb.coop Fri Aug 17 21:12:35 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:12:35 -0500 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <502E97A3.2090804@lcwb.coop> On 08/16/2012 09:21 AM, Jay K wrote: > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. In the stabs debug info given to (m3)gdb, they do have field names, with the original source code field name embedded within, plus a character-encoded uid and some flags, etc. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > This is the more reasonable representation, in view of the fact that any type expression is itself anonymous, as you note below, while giving it a type name is a separate matter, done outside the type definition. > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". Yes. > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > Yes. > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > This business of anonymous types is pervasive, not just for enumeration types and not just in Modula-3. It shows up all over the place in StubGen, which has lots of code obviously intended to handle anonymous object types and, not so obviously, broken, or at least unfinished. I guess nobody actually does it. Having once been inside that code, I would not define an anonymous type I wanted to have involved in anyway with network objects. Anonymous types are usually not particularly good coding style, and I almost always regret defining one and end up giving it a name after all, as I want to use it in multiple places. Modula-3's structural type equality rules perhaps encourage it more than so-called name equality. C encourages it too for the declarator types and by requiring manual construction of pass-by-reference by the programmer, instead of parameter modes. Something similar for the second-class treatment of arrays. > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > I don't remember the details, but m3gdb gets enumeration value names, displays them, and recognizes them in commands. They had to go through parse.c to get to stabs. > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > I know this is a broken record, but the fact that debug info comes off the side of parse.c means it can't reflect optimizations/transformations done later by the gcc backend. E.g. tree-nested.c. > - Jay From rodney_bates at lcwb.coop Fri Aug 17 21:28:53 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 17 Aug 2012 14:28:53 -0500 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: References: , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu> Message-ID: <502E9B75.8040100@lcwb.coop> On 08/15/2012 09:07 PM, Jay K wrote: > > There is a bootstrap issue here isn?t there? > > I am not sure. I don't understand the system enough. :( > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > The compiler does for example construct the type BOOLEAN out of lower level pieces. > Can it not then proceed to compute its typeid, and inform the backend about it? > > > Other examples appear to include...wait.where did I get this list? > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > I built up the following list: > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > UID_LONGINT = 16_05562176; (* LONGINT *) > UID_WORD = 16_97E237E2; (* CARDINAL *) > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > UID_REEL = 16_48E16572; (* REAL *) > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > UID_WIDECHAR = 16_88F439FC; > UID_MUTEX = 16_1541F475; (* MUTEX *) > UID_TEXT = 16_50F86574; (* TEXT *) > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > UID_ROOT = 16_9D8FB489; (* ROOT *) > UID_REFANY = 16_1C1C45E6; (* REFANY *) > UID_ADDR = 16_08402063; (* ADDRESS *) > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > UID_NULL = 16_48EC756E; (* NULL *) > > > Surely these aren't all fundamental?? > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > TYPE Record1 = RECORD REF Record2 END; > TYPE Record2 = RECORD REF Record1 END; > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive definitions and declarations, your M3 example being one. Modula-3's rule that all declarations in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, along with separate rules explicitly disallowing certain forms of recursive declarations that would be troublesome. E.g., deleting both occurrences of "REF" above would give records of infinite size. You might say the language uses "opt-out" for certain recursive declarations. Most languages use an opt-in strategy. The default rules is declare-before-use. For select kinds of declarations, there is some kind of "forward" declaration that declares the name and only some of its properties. After that, the name then can be used in limited ways (most interestingly, in other declarations). Eventually, there has to be a complete declaration that give the name again and supplies the remaining properties. After that it can be used in all legal ways. A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- use strategy with a system of forward declarations. My impression is that gcc is this way, but poorly documented. Will it give enough hints to just look at the front end output for selected recursive declaration sets like this example? It would at least give something known to be acceptable to gdb. Or is gdb changing the rules with new releases? > - Jay > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- > From: hosking at cs.purdue.edu > Date: Wed, 15 Aug 2012 19:58:23 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > Something seems off to me in the current implementation. > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > I hardcoded knowledge of them in parse.c and M3C.m3. > That seems wrong. > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > Shouldn't m3front inform the backend via m3cg of these types? > It is doable using the existing interfaces? > > > More so,RTBuiltin.mx ought not exist, right? > Whatever data it contains should be built up like any other type data? > Part of the same problem? > > > - Jay > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile+1 765 427 5484 > > > > > From rodney_bates at lcwb.coop Sat Aug 18 19:51:11 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 18 Aug 2012 12:51:11 -0500 Subject: [M3devel] reference to globals in globals? In-Reply-To: References: , <502BB77D.1030400@lcwb.coop>, , Message-ID: <502FD60F.7050002@lcwb.coop> On 08/15/2012 05:07 PM, Jay K wrote: > > If fact, if you are accustomed to thinking in type-safe terms > > > I do think in fairly type-terms. > In C++ I know to use templates, for example. > There is a matter of lifetime-safety though. > I understand that they are related -- if I get the lifetime wrong, then I end up with the type wrong, e.g. using freed-and-possibly-reallocated memory as if it was what it used to be and what I thought it still is. > > > Consider C++ code like this, where I want to do some things > "the same" to a bunch of variables, and some things "specific". > > > struct Person_t { const char* name; }; > > void F() > { > Person_t jay {"jay"}; > Person_t randy = {"randy"}; > Person_t tony = {"tony"}; > > // print all their names > Person_t* people[] = {&jay, &randy, &tony}; > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i]->name); > > DoSomethingSpecificToJay(&jay); > DoSomethingSpecificToRandy(&randy); > DoSomethingSpecificToTony(&tony); > > } > In Modula-3, you can code like this too, but you have to declare it UNSAFE and use unsafe constructs, just like you have to use in C, but don't have to admit explicitly . > I do NOT want to do this: > > void F() > { > Person_t people[3] = {{"jay"}, {"randy"}, {"tony"}}; > > > // print all their names > for (i = 0; i < sizeof(people) / sizeof(people[0]; ++i) > printf("%s\n", people[i].name); > > DoSomethingSpecificToJay(&people[0]); > DoSomethingSpecificToRandy(&people[1]); > DoSomethingSpecificToTony(&people[2]); > > } > > I understand there is: > enum People { jay, randy, tony }; > ... > DoSomethingSpecificToJay(&people[jay]); > DoSomethingSpecificToRandy(&people[randy]); > DoSomethingSpecificToTony(&people[tony]); > ... > > > but I'm still not thrilled about that. > > I ended up without anything "specific" except initialization..so no > really need to hang a name/variable on the data, and the > initialization is fairly specific, so the resulting code is merely: > > > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Integer.cg_type, typeid := UID_INTEGER)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word.cg_type, typeid := UID_WORD)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Int64.cg_type, typeid := UID_LONGINT)); > EVAL Type_Init(NEW(Integer_t, cg_type := Target.Word64.cg_type, typeid := UID_LONGWORD)); > > EVAL Type_Init(NEW(Float_t, cg_type := Target.Real.cg_type, typeid := UID_REEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Longreal.cg_type, typeid := UID_LREEL)); > EVAL Type_Init(NEW(Float_t, cg_type := Target.Extended.cg_type, typeid := UID_XREEL)); > > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_BOOLEAN, max := 1)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word8.cg_type, typeid := UID_CHAR, max := 16_FF)); > EVAL Type_Init(NEW(Enum_t, cg_type := Target.Word16.cg_type, typeid := UID_WIDECHAR, max := 16_FFFF)); > > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_31, min := 0, max := 31)); > EVAL Type_Init(NEW(Subrange_t, cg_type := Target.Integer.cg_type, typeid := UID_RANGE_0_63, min := 0, max := 31)); > > ... > > you can see it in m3back/src/M3C.m3 (which tangentially I think strikes at possible problems > in the m3cg interface...these types need to be defined via function calls into the backend, > not hardcoded..and even if they are...the order of types arriving at the backend isn't always > ideal..types maybe come in before they are referenced..maybe that is unavoidable due to loops... > I suspect I need to put in the multiple passes like I did in parse.c...we'll see...) > > > - Jay > > > > From: jay.krell at cornell.edu > > Date: Wed, 15 Aug 2012 12:56:09 -0400 > > To: hosking at cs.purdue.edu > > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] reference to globals in globals? > > > > I restructured the code but it still bothers me. Getting the levels of indirection correct is checked for you in C/C++ as '&' returns a stronger type than 'ADR'. > > I didn't want the array only because then I could only access the data more verbosely/slowly via the array. More later, maybe. > > > > - Jay (briefly/pocket-sized-computer-aka-phone) > > > > On Aug 15, 2012, at 12:11 PM, Tony Hosking wrote: > > > > > Jay, > > > > > > Any time you want to pass a reference to a local/global as a parameter you can use VAR/READONLY parameter mode. > > > > > > I don?t know enough about your use-case to understand what you are trying to do. > > > > > > > > > On Aug 15, 2012, at 10:51 AM, "Rodney M. Bates" wrote: > > > > > >> > > >> > > >> On 08/14/2012 10:04 PM, Jay K wrote: > > >>> Isn't it safe to take the address of a global? > > >>> > > >> > > >> Do you mean can't you use the ADR function in safe code > > >> if you apply it only to a global variable? The answer > > >> to that is no. The ADR function is illegal altogether in > > >> safe code. > > >> > > >> As to why, I can only speculate, but see below. I suspect > > >> even in this case, it is not as simple as it seems. > > >> > > >>> > > >>> I have something like this: > > >>> > > >>> > > >>> CONST UID_INTEGER = 1234; > > >>> CONST UID_FLOAT = 4567; > > >>> ... several more ... > > >>> > > >>> > > >>> TYPE CType = OBJECT .. END; > > >>> > > >>> > > >>> VAR t_int: CType := ...; > > >>> VAR t_float: CType := ...; > > >>> ... several more ... > > >>> > > >>> > > >>> MapTypeIdToType(UID_INTEGER, t_int); > > >>> MapTypeIdToType(UID_FLOAT, FLOAT); > > >>> ... several more ... > > >>> > > >>> > > >>> but what I really want is more like: > > >>> > > >>> > > >>> TYPE RECORD = BuiltinUid_t = > > >>> typeid: INTEGER; > > >>> ctype: REF CType; > > >> > > >> ^UNTRACED REF? If it were just REF, that would imply that > > >> your global variable (the pointer it contains) is a heap object, that > > >> it has heap allocator/GC overhead data attached to it, and that the GC > > >> should trace it, none of which is true. > > >> > > >> > > >>> END; > > >>> > > >>> > > >>> CONST BuiltinUids = ARRAY OF BuiltinUids { > > >>> BuiltinUids{UID_INTEGER, &t_int}, > > >>> BuiltinUids{UID_FLOAT, &t_float}, > > >> > > >> ADR instead of &? If so, you are still not there, because ADR > > >> returns a value of type ADDRESS, i.e., an untraced reference to > > >> we-don't-know-what. Somewhere, you would also have to use a > > >> LOOPHOLE to get it to UNTRACED REF CType. > > >> > > >>> ... several more ... > > >>> }; > > >>> > > >>> > > >>> FOR i := FIRST(BuiltinUids) TO LAST(BuiltinUids) DO > > >>> MapTypeIdToType(BuiltinUids[i].typeid, BuiltinUids[i].ctype); > > >>> END; > > >>> > > >> > > >> I don't know what the signature of MapTypeIdToType is, but above, > > >> you are passing a variable of object type to its 2nd parameter, > > >> (which contains a traced reference to the actual heap object). > > >> But here, you pass the _address_ of the above. Inconsistent > > >> number of levels of indirection. A static safe language is > > >> much more likely to help with things like this. > > >> > > >> Maybe you just want to say > > >> > > >> TYPE RECORD = BuiltinUid_t = > > >> typeid: INTEGER; > > >> ctype: CType; > > >> > > >> and > > >> > > >> BuiltinUids{UID_INTEGER, t_int}? > > >> > > >> This would be equivalent to your first way, and doesn't require any > > >> unsafe coding at all. > > >> > > >> Or, you could do away with global variable t_int altogether and > > >> just initialize directly into BuiltinUids[..].ctype with whatever > > >> expression you used to initialize t_int. It looks like your array > > >> makes the t_int and cousins redundant. > > >> > > >>> > > >>> Heck, even if these weren't global, is it that unreasonble, > > >>> from the programmer's point of view, for the language/compiler > > >>> to do some pointer escape analysis and let me take the address > > >>> of a local, as long as I don't store it somewhere that outlives > > >>> the local? > > >>> > > >> > > >> This is ultimately an undecidable problem and even conservative > > >> approximations of reasonable sophistication are far too involved > > >> for a language to require of every compiler. > > >> > > >>> > > >>> You can see this particular pattern currently in > > >>> m3-sys/m3cc/gcc/gcc/m3cg/parse.c > > >>> > > >>> > > >>> and I'm pretty busy now working on m3-sys/m3back/src/M3C.m3 > > >>> where I encounter this. > > >>> > > >>> > > >>> Working in safe languages can be frustrating... > > >>> > > >> > > >> It's just an instant/deferred gratification thing. Safe languages often > > >> make you stop and fix it before you run it. Unsafe languages let you naively > > >> forge ahead to the next step, where the bug is likely to be *much* harder to > > >> diagnose, assuming you even have enough test cases to notice it at all during > > >> development. Your code here is a good example. > > >> > > >> Of course, safe languages occasionally make you unnecessarily write a bit more > > >> code to do it the safe way. E.g., the famous fake pointer to the root of a > > >> linked list example. In my personal experience, these times are at least > > >> one order of magnitude less frequent than the times safe languages reduce > > >> great pain to minor pain, albeit sooner. If fact, if you are accustomed to > > >> thinking in type-safe terms, it is seldom any harder to code it safely in the > > >> first place. > > >> > > >> You're making it too difficult. > > >> > > >>> > > >>> Thank you, > > >>> - Jay > > >> > > > From jay.krell at cornell.edu Sun Aug 19 09:11:30 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:11:30 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? Message-ID: I propose the following changed to m3front, TInt, TFloat. It does kind of "infact" the front/middle end with the "intent" of a specific backend. When a TInt or TFloat is initialized from text, that text is remembered. When such a TInt/TFloat is added/subtracted/multiplied/divided by another, the obvious text is built up in the target. Initialization of TInt with constants could/should also maintain the textual representation. The implication is multi part: 1) init_float in a C-or-such backend can just output the textual expression. No need to worry about roundtripping it. 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" (preceded by a typedef for INTEGER in the larger context)..thereby factoring out the target from the generated C. 3) This provides some hypothetical portability when host/target don't use the "same" sort of floating point. Ignoring rounding details however. And TFloat does seem to round correctly at compile-time, assuming host/target are about the same -- and they all are. (What does the language definition say here anyway? I'll have to check..) Reasonable? For now I'll probably punt. I can just use TFloat.ToChars and munge it slightly. But this is probably a good direction. In the short term I'm trying hard to make do w/o m3front/m3middle changes. You know..implementing another backend, you tend to visit more of the system, gradually understand it..think of what you how one might prefer it to be..esp. for the task at hand. :) Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be?If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 19 09:14:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 07:14:23 +0000 Subject: [M3devel] m3 backends and builtin types? In-Reply-To: <502E9B75.8040100@lcwb.coop> References: , , <324320B1-3C69-46FE-8F4C-5452C3BA3FA5@cs.purdue.edu>, , <502E9B75.8040100@lcwb.coop> Message-ID: Thank for the pointers to m3gdb -- it kind of confirms my belief that the system isn't great.Even if there is a bootstrapping problem, it seems wrong for each backend and m3gdb to pay for it.Just m3front/m3front should have this hardcoded information. There is an m3front -emit option to ... i think provide the information..but it fails.I suspect it used and works when building just m3core.I have to look more. For now I'm punting on paying any attention to type information in the C-backend-in-progress.Just like the other backends ignore it all also.(ok..gcc uses it in the wierd way for m3gdb...) - Jay > Date: Fri, 17 Aug 2012 14:28:53 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3 backends and builtin types? > > > > On 08/15/2012 09:07 PM, Jay K wrote: > > > There is a bootstrap issue here isn?t there? > > > > I am not sure. I don't understand the system enough. :( > > > > > > I suspect, but am not certain, the ultimate source of this information is/can-be m3front/builtinTypes. > > The compiler does for example construct the type BOOLEAN out of lower level pieces. > > Can it not then proceed to compute its typeid, and inform the backend about it? > > > > > > Other examples appear to include...wait.where did I get this list? > > At some point, I was verifying that gcc/gcc/m3cg/parse.c was "informed" about every typeid that it later recieved a use of. > > Like, declare_array/declare_open_array/declare_enum/declare_typename should all precede declare_local, declare_param, declare_procedure (the return type). > > I built up the following list: > > > > > > UID_INTEGER = 16_195C2A74; (* INTEGER *) > > UID_LONGINT = 16_05562176; (* LONGINT *) > > UID_WORD = 16_97E237E2; (* CARDINAL *) > > UID_LONGWORD = 16_9CED36E7; (* LONGCARD *) > > UID_REEL = 16_48E16572; (* REAL *) > > UID_LREEL = 16_94FE32F6; (* LONGREAL *) > > UID_XREEL = 16_9EE024E3; (* EXTENDED *) > > UID_BOOLEAN = 16_1E59237D; (* BOOLEAN [0..1] *) > > UID_CHAR = 16_56E16863; (* CHAR [0..255] *) > > UID_WIDECHAR = 16_88F439FC; > > UID_MUTEX = 16_1541F475; (* MUTEX *) > > UID_TEXT = 16_50F86574; (* TEXT *) > > UID_UNTRACED_ROOT = 16_898EA789; (* UNTRACED ROOT *) > > UID_ROOT = 16_9D8FB489; (* ROOT *) > > UID_REFANY = 16_1C1C45E6; (* REFANY *) > > UID_ADDR = 16_08402063; (* ADDRESS *) > > UID_RANGE_0_31 = 16_2DA6581D; (* [0..31] *) > > UID_RANGE_0_63 = 16_2FA3581D; (* [0..63] *) > > UID_PROC1 = 16_9C9DE465; (* PROCEDURE (x, y: INTEGER): INTEGER *) > > UID_PROC2 = 16_20AD399F; (* PROCEDURE (x, y: INTEGER): BOOLEAN *) > > UID_PROC3 = 16_3CE4D13B; (* PROCEDURE (x: INTEGER): INTEGER *) > > UID_PROC4 = 16_FA03E372; (* PROCEDURE (x, n: INTEGER): INTEGER *) > > UID_PROC5 = 16_509E4C68; (* PROCEDURE (x: INTEGER; n: [0..31]): INTEGER *) > > UID_PROC6 = 16_DC1B3625; (* PROCEDURE (x: INTEGER; n: [0..63]): INTEGER *) > > UID_PROC7 = 16_EE17DF2C; (* PROCEDURE (x: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_PROC8 = 16_B740EFD0; (* PROCEDURE (x, y: INTEGER; i, n: CARDINAL): INTEGER *) > > UID_NULL = 16_48EC756E; (* NULL *) > > > > > > Surely these aren't all fundamental?? > > > > > > Eventually I decided that declare_array/typename/open_array/enum don't come in a nice order anyway, and I changed parse.c to loop until it could "resolve" all typeids -- loop such that declare_ preceeds declare_local/param/field. > > > > > > That still bugs me, but I haven't verified just what the front end is doing, and if there aren't ultimately possibly circularities that make a sort impossible anyway...consider: > > > > > > TYPE Record1 = RECORD REF Record2 END; > > TYPE Record2 = RECORD REF Record1 END; > > > > If there is only declare_record followed by declare_field, it will "never work" the way I wanted -- you'll always have to see a ref to a type before seeing the type defined. Ok. So I have to deal with an arbitrary order.. I guess I'll go and write the required loop -- and hold everything in memory... at least for a later pass at this stuff. The initial version doesn't need great type info, just as the existing backends don't need it...eh..I guess I can do without fixing the original issue first therefore.. > > > > > > This is a fundamental problem. There are ways perfectly legitimate programs need recursive > definitions and declarations, your M3 example being one. Modula-3's rule that all declarations > in a scope are made "simultaneously" is an unusual (maybe unique) way of supporting this, > along with separate rules explicitly disallowing certain forms of recursive declarations > that would be troublesome. E.g., deleting both occurrences of "REF" above would give > records of infinite size. You might say the language uses "opt-out" for certain recursive > declarations. > > Most languages use an opt-in strategy. The default rules is declare-before-use. For select > kinds of declarations, there is some kind of "forward" declaration that declares the name > and only some of its properties. After that, the name then can be used in limited ways (most > interestingly, in other declarations). Eventually, there has to be a complete declaration that > give the name again and supplies the remaining properties. After that it can be used in all > legal ways. > > A lot of intermediate languages, e.g. the input language(s) of gdb tend to use the declare-before- > use strategy with a system of forward declarations. My impression is that gcc is this way, > but poorly documented. > > Will it give enough hints to just look at the front end output for selected recursive declaration > sets like this example? It would at least give something known to be acceptable to gdb. Or > is gdb changing the rules with new releases? > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > -- > > From: hosking at cs.purdue.edu > > Date: Wed, 15 Aug 2012 19:58:23 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3 backends and builtin types? > > > > There is a bootstrap issue here isn?t there? You need the compiler to inform the runtime of information that the compiler needs... > > > > On Aug 15, 2012, at 6:11 PM, Jay K > wrote: > > > > Something seems off to me in the current implementation. > > Like, I don't think the backends are ever informed of various "builtin" types, such as integer, word, char, widechar, boolean, mutex. > > I hardcoded knowledge of them in parse.c and M3C.m3. > > That seems wrong. > > > > > > Either that, or they are used before they are defined -- which might not be avoidable in general, but could easily be avoided for most types. > > > > > > Shouldn't m3front inform the backend via m3cg of these types? > > It is doable using the existing interfaces? > > > > > > More so,RTBuiltin.mx ought not exist, right? > > Whatever data it contains should be built up like any other type data? > > Part of the same problem? > > > > > > - Jay > > > > > > > > > > Antony Hosking|Associate Professor | Computer Science | Purdue University > > 305 N. University Street | West Lafayette | IN 47907 | USA > > Mobile+1 765 427 5484 > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 19 17:45:00 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 19 Aug 2012 11:45:00 -0400 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <20120819154500.GA17607@topoi.pooq.com> On Sun, Aug 19, 2012 at 07:11:30AM +0000, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. How many digits are you planning to keep for 1.0/3.0 ? -- hendrik From rodney_bates at lcwb.coop Sun Aug 19 23:48:27 2012 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 19 Aug 2012 16:48:27 -0500 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: References: Message-ID: <50315F2B.2060300@lcwb.coop> On 08/19/2012 02:11 AM, Jay K wrote: > I propose the following changed to m3front, TInt, TFloat. > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > When a TInt or TFloat is initialized from text, that text is remembered. > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > the obvious text is built up in the target. > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > The implication is multi part: > > > 1) init_float in a C-or-such backend can just output the textual expression. > No need to worry about roundtripping it. > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > out the target from the generated C. > > > 3) This provides some hypothetical portability when host/target > don't use the "same" sort of floating point. Ignoring rounding details however. > And TFloat does seem to round correctly at compile-time, assuming host/target > are about the same -- and they all are. > (What does the language definition say here anyway? I'll have to check..) > As far as I can find, the language says nothing about this. However, I feel *extremely* strongly that arithmetic done at compile time should be *exactly* the same as on the target machine, including what happens in exceptional cases, e.g., overflows, NaNs, etc. I know this can be hard to get right, and it raises hard questions about division of work between front/back. Even harder when there are multiple back ends, especially ones you don't control. But I consider it a serious compiler bug if CT and RT arithmetic differ. This might be something that could be postponed for more urgent things, but the design at least should anticipate it. I have no problems with the principal of preserving/generating textual representations. However, if you really implement target arithmetic somewhere, it might be easier to convert back and forth than to do faithful target arithmetic on character strings. If that is what you are referring to as {not} roundtripping. > > Reasonable? > > > For now I'll probably punt. > I can just use TFloat.ToChars and munge it slightly. > But this is probably a good direction. > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > You know..implementing another backend, you tend to visit more of the system, > gradually understand it..think of what you how one might prefer it to be..esp. > for the task at hand. :) > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > - Jay > > From jay.krell at cornell.edu Mon Aug 20 00:08:26 2012 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Aug 2012 22:08:26 +0000 Subject: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? In-Reply-To: <50315F2B.2060300@lcwb.coop> References: , <50315F2B.2060300@lcwb.coop> Message-ID: > However, if you really implement target arithmetic somewhere, I don't. I just want to print out constants or expressions for a C or C++ compiler to deal with. My initial goal is a C-generating backend that "work". The code will be target-dependent and not very debuggable (like what stock debuggers do today with Modula-3). Longer term, I want the code to be target-endian-indepenent, target-wordsize-independent, target-jmpbuf-independent, and possibly floating-point-representation-independent -- completely target-independent. Floating-point-representation independence doesn't gain anything -- all hosts and targets use the same representation, and I think the current system depends on that anyway (see TFloat). Floating-point-representation independence is "difficult" for "most compilers", but if generating C/C++, I think it is easy. target-wordsize-independence I believe shall be gained by my proposal here. target-endian-indepencence I believe shall be gained by small changes in m3front and the existing backends -- m3front only checks endianness in like one or two places. target-jmpbuf-size-independence I believe shall be gained by roughly the change I made in late 2010, but w/o causing loops to allocate stack for every try "invocation". Or, well, there is likely another way -- some indication to the backend that the variable is a jumpbuf and then outputing "jmp_buf jb" in the C backend..it should be easy enough..add declare_jmpbuf that has the same signature as declare_local, or add a boolean to declare_local. That is more efficient than the alloca(Csetjmp_jmpbuf_size) approach, but doesn't achieve the same gain for m3middle -- that is, even when targeting m3cc, m3front/m3middle ought not have this information -- that would cut out yet one more target-dependence in cm3/m3front/m3middle. Anyway, I'm not to this point yet. I have a fair amount more to do just for target-dependent C to be generated and work. I have made progress, but there is more to do. - Jay > Date: Sun, 19 Aug 2012 16:48:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3front/TInt/TFloat should maintain "textual expressions" for constants? > > > > On 08/19/2012 02:11 AM, Jay K wrote: > > I propose the following changed to m3front, TInt, TFloat. > > It does kind of "infact" the front/middle end with the "intent" of a specific backend. > > > > > > When a TInt or TFloat is initialized from text, that text is remembered. > > When such a TInt/TFloat is added/subtracted/multiplied/divided by another, > > the obvious text is built up in the target. > > > > > > Initialization of TInt with constants could/should also maintain the textual representation. > > > > > > The implication is multi part: > > > > > > 1) init_float in a C-or-such backend can just output the textual expression. > > No need to worry about roundtripping it. > > > > > > 2) Longer term, TInt "constant expressions" could contain like "sizeof(INTEGER)" > > (preceded by a typedef for INTEGER in the larger context)..thereby factoring > > out the target from the generated C. > > > > > > 3) This provides some hypothetical portability when host/target > > don't use the "same" sort of floating point. Ignoring rounding details however. > > And TFloat does seem to round correctly at compile-time, assuming host/target > > are about the same -- and they all are. > > (What does the language definition say here anyway? I'll have to check..) > > > > As far as I can find, the language says nothing about this. > > However, I feel *extremely* strongly that arithmetic done at > compile time should be *exactly* the same as on the target machine, > including what happens in exceptional cases, e.g., overflows, > NaNs, etc. > > I know this can be hard to get right, and it raises hard questions > about division of work between front/back. Even harder when there > are multiple back ends, especially ones you don't control. > But I consider it a serious compiler bug if CT and RT arithmetic > differ. This might be something that could be postponed for more > urgent things, but the design at least should anticipate it. > > I have no problems with the principal of preserving/generating > textual representations. However, if you really implement target > arithmetic somewhere, it might be easier to convert back and forth > than to do faithful target arithmetic on character strings. If > that is what you are referring to as {not} roundtripping. > > > > > Reasonable? > > > > > > For now I'll probably punt. > > I can just use TFloat.ToChars and munge it slightly. > > But this is probably a good direction. > > In the short term I'm trying hard to make do w/o m3front/m3middle changes. > > > > > > You know..implementing another backend, you tend to visit more of the system, > > gradually understand it..think of what you how one might prefer it to be..esp. > > for the task at hand. :) > > > > > > Obviously this is somewhat of a hack, in that...what should the syntax of the expressions be? > > If/when I make the change, I'll probably use C. It will be almost identical to Modula-3 anyway. > > > > > > - Jay > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Aug 21 13:18:48 2012 From: dmuysers at hotmail.com (Dirk Muysers) Date: Tue, 21 Aug 2012 13:18:48 +0200 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: *** A warning *** Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: Code generation is my business :-) Comments on a few options: a.. CLR: a.. Pro: industrial support b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter c.. Con: Only Windows platform is really prime-time quality b.. LLVM: a.. Pro: enthusiastic user community with charismatic leader b.. Pro: serious backing from Apple c.. Pro: many interesting performance improvements d.. Con: somewhat complex interface e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface c.. C-- a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code b.. Pro: design is reasonably mature and reasonably clean c.. Pro: supports accurate garbage collection d.. Pro: most users report it is very easy to use e.. Con: very small development team f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) g.. Con: does not ship with a garbage collector h.. Con: project has no future d.. C as target language a.. Pro: looks easy b.. Con: nearly impossible to get decent performance c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project. From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 14:33:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 13:33:27 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: Message-ID: <1345552407.75520.YahooMailClassic@web29704.mail.ird.yahoo.com> Hi all: It might need some pre-processor but C is supposedly superior in performance than C# and Java by far, I mean we are not talking about using C, we are talking about generating object code from UNSAFE code, but it shouldn't be that bad in terms of assembly level compile and link time process compared to those systems, if not then why every body wants their software be written in (e.g us) Modula-3, but compiled in a C environment it must be something about engineering causes of who wrote that thing first, and C is much older, so there is better support for that UNSAFE level code (instead look C3 and Java, lame), and that's for me more important in this matter. Probably C is referred as a high-level assembly (like DEC-SRC first versions), instead of a low-level assembly language like here we are trying to do. In fact if we were able to generate C from M3CG we can gain better portability, for instance my project of compare cross-boots would be more easily than by itself now. Thanks in advance --- El mar, 21/8/12, Dirk Muysers escribi?: De: Dirk Muysers Asunto: Re: [M3devel] higher level m3cg? Para: "Jay K" , "m3devel" Fecha: martes, 21 de agosto, 2012 06:18 *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 21 16:10:02 2012 From: hosking at cs.purdue.edu (Antony Hosking) Date: Tue, 21 Aug 2012 10:10:02 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr. Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > CLR: > > Pro: industrial support > Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > Con: Only Windows platform is really prime-time quality > LLVM: > > Pro: enthusiastic user community with charismatic leader > Pro: serious backing from Apple > Pro: many interesting performance improvements > Con: somewhat complex interface > Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface > C-- > > Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > Pro: design is reasonably mature and reasonably clean > Pro: supports accurate garbage collection > Pro: most users report it is very easy to use > Con: very small development team > Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > Con: does not ship with a garbage collector > Con: project has no future > C as target language > > Pro: looks easy > Con: nearly impossible to get decent performance > Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Aug 21 16:53:37 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 15:53:37 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <1345560817.79160.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: target dependency is essential in this process, so a good backend must target an quintessential machine, so I don't agree a given machine will give us lot of optimization without lot of work: http://irclog.perlgeek.de/parrot/2010-07-25#i_2605776 M3CG can be converted to a virtual machine as an emulator of an automaton, but studying that automaton complexity will give us the real facts (instead on believing stories), I don't want to fight wars among languages, but by practice we now know that C is far easier to code low level stuff than everything else, so .. if we want a radical change let's study why we do it first. Thanks in advance --- El mar, 21/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: "m3devel" , "Jay K" Fecha: martes, 21 de agosto, 2012 09:10 I concur strongly with Norman's analysis. We should be focusing on llvm. That will still require lifting the level of m3cg slightly, viz getelementptr.? Sent from my iPhone On Aug 21, 2012, at 7:18, "Dirk Muysers" wrote: *** A warning *** Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: ? Code generation is my business :-) Comments on a few options: CLR: Pro: industrial support Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter Con: Only Windows platform is really prime-time quality LLVM: Pro: enthusiastic user community with charismatic leader Pro: serious backing from Apple Pro: many interesting performance improvements Con: somewhat complex interface Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface C-- Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code Pro: design is reasonably mature and reasonably clean Pro: supports accurate garbage collection Pro: most users report it is very easy to use Con: very small development team Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) Con: does not ship with a garbage collector Con: project has no future C as target language Pro: looks easy Con: nearly impossible to get decent performance Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. Full disclosure: I am affiliated with the C-- project.? From: Jay K Sent: Thursday, August 16, 2012 4:21 PM To: m3devel Subject: [M3devel] higher level m3cg? Should m3cg provide enough information for a backend to generate idiomatic C? (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) Should we make it so? Or be pragmatic and see if anyone gets to that point? But, look at this another way. Let's say we are keeping the gcc backend. Isn't it reasonable to have a better experience with stock gdb? What should m3cg look like then? Matching up m3front to gcc turns out to be "wierd". As does having a backend generate "C". In particular, "wierd" because there is a "level mismatch". m3cg presents a fairly low level view of the program. ? It does layout. Global variables are stuffed into what you might call a "struct", with no assigned field names. Field references are done by adding to addresses and casting. Too low level to provide a "good" gcc tree representation or to generate "normal" C. One might be able to, by somewhat extraordinary means, make due. That is, specifically one could deduce field references from offsets/sizes. But maybe it is reasonable for load/store to include fields? Maybe in addition to what it provides? As well, it appears to me, that given TYPE Enum = {One, Two, Three}; the m3cg is like: declare enum typeidblah declare enum_elt One declare enum_elt Two declare enum_elt Three declare_typename typeidblah Enum One kind of instead wants more like: declare enum typeidblah Enum declare enum_elt One => rename it Enum_One declare enum_elt Two "" declare enum_elt Three "" However I understand that {One, Two, Three} exists as anonymous type independent of the name "Enum". One could just as well have: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = {One, Two, Three}; Enum1 and Enum2 probably have the same typeid, and are just two typenames for the same type. likewise: given TYPE Enum1 = {One, Two, Three}; given TYPE Enum2 = Enum1; but, pragmatically, in the interest of generating better C, can we pass a name along with declare_enum? I ask somewhat rhetorically. I realize there is the answer: ? enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; ? typedef enum Mtypeid Enum1; Also, enum variables I believe end up as just UINT8, 16, or 32. Loads of enum values I believe end up as just loads of integers. Can we pass along optional enum names with declare_local/declare_param? And optional enum names with load_int? Or add a separate load_enum call? Really, I understand that the current interface can be pressed to do pretty adequate things. I can infer field references. The way enums work isn't too bad. ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Tue Aug 21 20:14:24 2012 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Tue, 21 Aug 2012 20:14:24 +0200 (CEST) Subject: [M3devel] higher level m3cg? In-Reply-To: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: On Tue, 21 Aug 2012, Antony Hosking wrote: > *** A warning *** > Norman Ramsey's opinion?(in stackoverflow) on possible compiler backends: > ? > > Code generation is my business :-) > > Comments on a few options: > > * > > CLR: > > + Pro: industrial support > + Con: you have to buy into their type system pretty much completely; depending on what you want > to do with types, this may not matter > + Con: Only Windows platform is really prime-time quality > * > > LLVM: > > + Pro: enthusiastic user community with charismatic leader > + Pro: serious backing from Apple > + Pro: many interesting performance improvements > + Con: somewhat complex interface > + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > to be plugged by adding to the complexity of the interface Additional Con: They introduce bugs from time to time and then they do not fix them ... > * > > C-- > > + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > your C-- code LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > + Pro: design is reasonably mature and reasonably clean > + Pro: supports accurate garbage collection > + Pro: most users report it is very easy to use > + Con: very small development team > + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > + Con: does not ship with a garbage collector > + Con: project has no future > * > > C as target language > > + Pro: looks easy > + Con: nearly impossible to get decent performance > + Con: will drive you nuts in the long run; ask the long line of people who have tried to > compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > of these people gave up and built their own native code generator. > > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > and expected longevity, I'd probably recommend LLVM. Despite my 'con' point I would recommend LLVM, too. From jay.krell at cornell.edu Tue Aug 21 21:26:49 2012 From: jay.krell at cornell.edu (Jay) Date: Tue, 21 Aug 2012 12:26:49 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> Message-ID: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >> *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >> >> Code generation is my business :-) >> Comments on a few options: >> >> * >> CLR: >> >> + Pro: industrial support >> + Con: you have to buy into their type system pretty much completely; depending on what you want >> to do with types, this may not matter >> + Con: Only Windows platform is really prime-time quality >> * >> LLVM: >> >> + Pro: enthusiastic user community with charismatic leader >> + Pro: serious backing from Apple >> + Pro: many interesting performance improvements >> + Con: somewhat complex interface >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >> to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >> C-- >> >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >> your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >> + Pro: design is reasonably mature and reasonably clean >> + Pro: supports accurate garbage collection >> + Pro: most users report it is very easy to use >> + Con: very small development team >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >> + Con: does not ship with a garbage collector >> + Con: project has no future >> * >> C as target language >> >> + Pro: looks easy >> + Con: nearly impossible to get decent performance >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >> of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > From dabenavidesd at yahoo.es Tue Aug 21 23:58:52 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 21 Aug 2012 22:58:52 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: <1345586332.94616.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: at the core of this issue, and if you want to ellucidate more, please C Ramsey, Lambda-RTL, and Necula Lambda-CIL respectively: http://terohasu.net/desclang/Description_languages.html#LAMBDA-RTL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.72.4008 Baby Modula-3 permits axiomatic reasoning over objects so it should be possible to reason over given program an its execution in a system of given semantics Thanks in advance --- El mar, 21/8/12, Jay escribi?: De: Jay Asunto: Re: [M3devel] higher level m3cg? Para: "Henning Thielemann" CC: "m3devel" , "JayK" Fecha: martes, 21 de agosto, 2012 14:26 I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > On Tue, 21 Aug 2012, Antony Hosking wrote: > >>? ? ? *** A warning *** >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: >>? >> Code generation is my business :-) >> Comments on a few options: >> >> * >>? ? CLR: >> >>? ???+? Pro: industrial support >>? ???+? Con: you have to buy into their type system pretty much completely; depending on what you want >>? ? ? ? to do with types, this may not matter >>? ???+? Con: Only Windows platform is really prime-time quality >> * >>? ? LLVM: >> >>? ???+? Pro: enthusiastic user community with charismatic leader >>? ???+? Pro: serious backing from Apple >>? ???+? Pro: many interesting performance improvements >>? ???+? Con: somewhat complex interface >>? ???+? Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering >>? ? ? ? to be plugged by adding to the complexity of the interface > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > >> * >>? ? C-- >> >>? ???+? Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit >>? ? ? ? your C-- code > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > >>? ???+? Pro: design is reasonably mature and reasonably clean >>? ???+? Pro: supports accurate garbage collection >>? ???+? Pro: most users report it is very easy to use >>? ???+? Con: very small development team >>? ???+? Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >>? ???+? Con: does not ship with a garbage collector >>? ???+? Con: project has no future >> * >>? ? C as target language >> >>? ???+? Pro: looks easy >>? ???+? Con: nearly impossible to get decent performance >>? ???+? Con: will drive you nuts in the long run; ask the long line of people who have tried to >>? ? ? ? compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one >>? ? ? ? of these people gave up and built their own native code generator. >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, >> and expected longevity, I'd probably recommend LLVM. > > Despite my 'con' point I would recommend LLVM, too. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 00:01:23 2012 From: jay.krell at cornell.edu (Jay K) Date: Tue, 21 Aug 2012 22:01:23 +0000 Subject: [M3devel] higher level m3cg? In-Reply-To: <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> References: <87A88370-378A-49B1-90B8-1446F062FEA4@cs.purdue.edu> , <37C6DBC1-0171-4241-8908-00C8124ED5BA@gmail.com> Message-ID: I find Ramsey's analysis rather incomplete. He mentions CLR has industrial strength support. But he fails to say that C also does, way more significantly so. And you can barely mention CLR w/o also mentioning Java, though CLR is more general. I know C has downsides. C, without C++, has the same poor exception handling. The performance of the compiler indeed will not be the best. The portability however will be tremendous. Basically every system I can think of except Windows Phone 7 and web browsers have a production quality C and C++ compiler. Examples of systems we don't support or support well today that a C/C++ backend would trivially enable, that are still manufactured, sold, updated: PPC{32,64}_AIX (still might need #ifdef in m3core) PPC64_{FREEBSD?,LINUX} ARM32_DARWIN (iPhone, iPad) MIPS64_{LINUX,OPENBSD} IA64_{VMS,HPUX,FREEBSD,LINUX} (still might need #ifdef in m3core due to the "two stacks"; I'm also worried that a 64bit integer closure marker is too small) ARM32_LINUX (including Android) MIPS32_LINUX (network routers!) ARM32_NT ARM64_? {PPC32,MIPS32,ARM32,I386}_CE plus some more obsolete systems: ALPHA{32,64}_VMS PPC64_DARWIN MIPS{32,64}_IRIX PA{32,64}_HPUX Granted, gcc/m3cc is *almost* as portable, and some of the above I have gotten far along in supporting -- really, given the gcc/m3cc/posix base, the biggest task to any port is getting the hardware up and running and with a decent C toolset. Plus that we would gain a "portable distribution format" that people could just tar xfz; cd; ./configure; make; make install Granted, I don't have much experience in any area here, and I don't yet have C working. LLVM will also take more time and learning. I probably won't put the time into it. I am hoping to get the C backend working, so that the cm3 system might be considered significantly more "done", so as new processors come along, no new porting will be needed, just as for the vast majority of code out there. LLVM supports many more architectures than listed below. Anyway..later.. - Jay > CC: hosking at cs.purdue.edu; dmuysers at hotmail.com; m3devel at elegosoft.com; jay.krell at cornell.edu > From: jay.krell at cornell.edu > Subject: Re: [M3devel] higher level m3cg? > Date: Tue, 21 Aug 2012 12:26:49 -0700 > To: lemming at henning-thielemann.de > > I don't see why C will be a problem. Performance of the compiler or the generated code? Our current system is bad on both fronts already, but nobody seems to notice. And yes it does look easy. I don't know why it'd cause craziness. M3cc is hardly writing our own & has serious shortcomings: stock debugging, exception handling... > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 21, 2012, at 11:14 AM, Henning Thielemann wrote: > > > > > On Tue, 21 Aug 2012, Antony Hosking wrote: > > > >> *** A warning *** > >> Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > >> > >> Code generation is my business :-) > >> Comments on a few options: > >> > >> * > >> CLR: > >> > >> + Pro: industrial support > >> + Con: you have to buy into their type system pretty much completely; depending on what you want > >> to do with types, this may not matter > >> + Con: Only Windows platform is really prime-time quality > >> * > >> LLVM: > >> > >> + Pro: enthusiastic user community with charismatic leader > >> + Pro: serious backing from Apple > >> + Pro: many interesting performance improvements > >> + Con: somewhat complex interface > >> + Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering > >> to be plugged by adding to the complexity of the interface > > > > Additional Con: They introduce bugs from time to time and then they do not fix them ... > > > > > >> * > >> C-- > >> > >> + Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit > >> your C-- code > > > > LLVM has also a text representation for its intermediate code. This way you can write intermediate code by hand and experiment with it. However, it is verbose and redundant. > > > >> + Pro: design is reasonably mature and reasonably clean > >> + Pro: supports accurate garbage collection > >> + Pro: most users report it is very easy to use > >> + Con: very small development team > >> + Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > >> + Con: does not ship with a garbage collector > >> + Con: project has no future > >> * > >> C as target language > >> > >> + Pro: looks easy > >> + Con: nearly impossible to get decent performance > >> + Con: will drive you nuts in the long run; ask the long line of people who have tried to > >> compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one > >> of these people gave up and built their own native code generator. > >> Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, > >> and expected longevity, I'd probably recommend LLVM. > > > > Despite my 'con' point I would recommend LLVM, too. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 22 09:28:50 2012 From: wagner at elego.de (Olaf Wagner) Date: Wed, 22 Aug 2012 09:28:50 +0200 Subject: [M3devel] Fw: SW Bug #1080 Message-ID: <20120822092850.cef2e188.wagner@elego.de> I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system. I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From mika at async.caltech.edu Wed Aug 22 13:00:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 04:00:48 -0700 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822110048.903811A207D@async.async.caltech.edu> In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > > a.. CLR:=20 > > a.. Pro: industrial support=20 > b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader=20 > b.. Pro: serious backing from Apple=20 > c.. Pro: many interesting performance improvements=20 > d.. Con: somewhat complex interface=20 > e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 > b.. Pro: design is reasonably mature and reasonably clean=20 > c.. Pro: supports accurate garbage collection=20 > d.. Pro: most users report it is very easy to use=20 > e.. Con: very small development team=20 > f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 > g.. Con: does not ship with a garbage collector=20 > h.. Con: project has no future > d.. C as target language > > a.. Pro: looks easy=20 > b.. Con: nearly impossible to get decent performance=20 > c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 From jay.krell at cornell.edu Wed Aug 22 15:39:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:39:28 +0000 Subject: [M3devel] suggest fixed size integers in m3cg_binrd/binwr instead of compact encoding, for trivial decoding/looping? Message-ID: BinRd/BinWr "redesign". How about BinRd/BinWr is "redesigned" with the following in mind: instead of most compact encoding, instead use least-decoding-needed encoding, which is also least compact in particular, store all integers as native-endian 8 byte integers include at the start of the stream, like: UINT8 file_endian; UINT64 integer_map_offset; UINT64 integer_map_size; reader would mmap the file (mmap has its pluses and minuses). The data would be presumed to fit in address space -- even if it tends to be accessed sequentially, random access would be possible. if file_endian != host_endian, it would jump ahead to the positions indicated by integer_map_offset and swap them. writer would note all these offsets while writing If I made this change, then I'd be much more likely to just use this "pre-"existing" format as an in-memory format that I loop over multiple times. Given that these are temporary files that don't usually make it to disk, optimizing for size isn't clearly valuable. Given this change, walking the m3cg data multiple times would end up just being bumping a pointer through an array of variably-sized records, the sizes of which would be trivially looked up based on the first byte (with the exception of case_jump). This would likely obvious my ongoing little M3CG_MultiPass tedious work. Something I believe is desired/needed here for M3C.m3. This isn't meant to be a pointless excercise (though it is *somewhat* speculative). m3front/m3cg/m3cg_ops would be unchanged. The change would be in M3CG_BinRd.m3/M3CG_BinWr.m3/parse.c. Thoughts? I've omitted the obvious detail -- floats/doubles. There'd actually be an array of MultiByteValueMaps. They'd include a size. There'd be "4" for floats, and "8" for INTEGERs and doubles. And hypothetically 10 or 16 for "extended", but those don't really exist. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 15:36:28 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 13:36:28 +0000 Subject: [M3devel] m3cg "Proc" and "Var" Message-ID: I'm wondering if it might just be much easier if m3front assigns small incrementing integers for Var and Proc rather than getting pointers back from declare_procedure, declare_local, etc., and then passing them back to begin_procedure, load, etc. See, where I am at is I want to "playback" an m3cg run multiple times, computing different data each time -- later runs depending on earlier runs. I don't need to a persisent "disk" form. Though I end up close to that, esp. with this proposed change, which eliminates some pointers. I already do this in parse.c. Parse.c being written in C++ and having the preprocessor, it was much easier to do. See how I #include m3-def.h multiple times, with different macros. Ugly perhaps but highly convenient and productive and easy to use.. I'm so tempted to write "m3toc" this way, but I'm trying to use the product. It is painful.. Anyway, so I'm using Modula-3, and I have this really tedious code to convert the m3cg calls into an in-memory format. That I'll then loop over multiple times. Due to the tedium, I build up the in-memory format in one shared module, that I'm calling M3CG_MultiPass. It is closely analogous to M3CG_Ops.i3: TYPE declare_exception_t = REF RECORD name: Name; arg_typeid: TypeUID; raise_proc: BOOLEAN; base: Var; offset: INTEGER END; PROCEDURE set_runtime_proc(self: T; name: Name; proc: Proc) = BEGIN self.Add(NEW(set_runtime_proc_t, name := name, proc := proc)); END set_runtime_proc; etc. so, now, for this in-memory format to be used by arbitrary other backends..I think these "Var" and "Proc" will become a sticking point. I think we need a backend-independent way to refer to them. I think INTEGER is the way. Like how M3CG_BinWr deals with them. The downside/cost is that in M3x86.m3, rather than directly using the pointer (after narrow), it will instead index into its own private array. For efficiency, maybe some hint would come to the backend early as to how large to make the arrays? Or maybe that defeats some of the design -- the single pass implementations. (m3front is many-pass anyway, right? All this single-pass stuff is kind of only theoretical, right? The compiler working-set is not small and fixed size, but rather scales with module size, right?) Anyway, ok, either way. Backends can just at-least-double their internal arrays as they go. One might suggest some sort of "multiple playback" based on M3CG_BinWr/BinRd. I'm open to that. But it has some extra unnneeded overhead vs. a typical in-memory representation. I'm also open to changing BinWr/BinRd to use a larger representation that requires "no" decoding. (I've sketched this out myself, but decided it is not a priority. BinWr/BinRd really only exist for licensing reasons and would likely go away if m3cc went away. I think. They are kind of otherwise useful, at least during development.) Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 22 16:24:42 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:24:42 +0000 Subject: [M3devel] why this loop/multipass stuff Message-ID: why this loop/multipass stuff So people understand where I'm going here -- and Tony asked. I had already written this up.. 1) in the past working on m3cc/parse.c, I found that "declare"s weren't always "topologically sorted". That is, like, I might see: given TYPE T1 = INTEGER; TYPE T2 = RECORD = a:T1; END; I might have seen type T1 declared before T2, or not. I'm not sure if it was record/fields or other relationships. I just remember that building up a map of typeid to data, and looking up it when needed, didn't always work, with one pass. So what I did is that I was willing to "skip" declares that referred to typeids I hadn't seen yet, and then loop again, hoping for progress at each loop (fewer skips). 2) It looks like declare_local/temp don't all come in "at the start" of a procedure. I'm not sure if I'm generating C or C++, but if I'm generating C, it helps to have declare_local/temp right after begin_procedure/block. Multiple passes let me make that so, without changing m3front. (Upon further looking..it seems this is mostly done right, but not for "try" blocks; maybe reasonable to change m3front here.). Now, generating C++ is an option, and I'm ignoring all declares currently anyway, so these don't force the matter, but: 3) nested functions. If a function contains any nested functions, I want to put all of its locals in a local struct, and use the address of that struct as the "static link". Or at least the locals that are referenced by nested functions. As well, m3front contains options as to the order of outputing functions. It uses one order for m3back and a different order for m3cc. I wouldn't mind if M3C.m3 worked either way. There are possibly other benefits that could be derived from multiple passes. Nothing offhand though. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Aug 22 16:28:44 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 22 Aug 2012 10:28:44 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 22, 2012, at 10:24 AM, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Was this all in aid of building up proper type descriptors? > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. Preferably these only, or else you will cause all locals to be in memory instead of registers. > As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 22 16:41:09 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 22 Aug 2012 07:41:09 -0700 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822144109.43DA71A2096@async.async.caltech.edu> Jay I've been following what you've been writing with one eye. But I'm not clear exactly what framework you're fitting into. Are you processing the output of m3front? There's always m3tk. Since it is claimed it was used to write a compiler it ought to be able to process MODULEs but I've only ever used it to process INTERFACEs. What's the story about that? Mika Tony Hosking writes: > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=iso-8859-1 > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > >> why this loop/multipass stuff >>=20 >>=20 >> So people understand where I'm going here -- and Tony asked. I had = >already written this up.. >>=20 >>=20 >>=20 >> 1) in the past working on m3cc/parse.c, I found that "declare"s >> weren't always "topologically sorted". >>=20 >>=20 >> That is, like, I might see: >>=20 >> given TYPE T1 =3D INTEGER; >> TYPE T2 =3D RECORD =3D a:T1; END; >>=20 >>=20 >> I might have seen type T1 declared before T2, or not. >> I'm not sure if it was record/fields or other relationships. >> I just remember that building up a map of typeid to data, and >> looking up it when needed, didn't always work, with one pass. >>=20 >>=20 >> So what I did is that I was willing to "skip" declares >> that referred to typeids I hadn't seen yet, and then loop >> again, hoping for progress at each loop (fewer skips). > >Was this all in aid of building up proper type descriptors? > >> 2) It looks like declare_local/temp don't all come in >> "at the start" of a procedure. I'm not sure if I'm generating >> C or C++, but if I'm generating C, it helps to have declare_local/temp >> right after begin_procedure/block. Multiple passes let me make that >> so, without changing m3front. >> (Upon further looking..it seems this is mostly done right, but >> not for "try" blocks; maybe reasonable to change m3front here.). >>=20 >>=20 >> Now, generating C++ is an option, and I'm ignoring all declares >> currently anyway, so these don't force the matter, but: >>=20 >>=20 >> 3) nested functions. >> If a function contains any nested functions, I want to put all of >> its locals in a local struct, and use the address of that struct >> as the "static link". Or at least the locals that are referenced >> by nested functions. > >Preferably these only, or else you will cause all locals to be in memory = >instead of registers. > >> As well, m3front contains options as to >> the order of outputing functions. It uses one order for m3back and >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either >> way. >>=20 >>=20 >> There are possibly other benefits that could be derived from multiple = >passes. >> Nothing offhand though. >>=20 >>=20 >> - Jay > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=iso-8859-1 > >charset=3Diso-8859-1">style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = >font-family: Helvetica; font-size: 12px; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; -webkit-text-decorations-in-effect: none; = >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = >font-style: normal; font-variant: normal; font-weight: normal; = >letter-spacing: normal; line-height: normal; = >-webkit-text-decorations-in-effect: none; text-indent: 0px; = >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = >white-space: normal; widows: 2; word-spacing: 0px; ">
class=3D"Apple-style-span" color=3D"#0000FF">class=3D"Apple-style-span" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Antony = >Hoskingface=3D"Gill Sans">'Gill Sans'; ">'Gill Sans'; "> |class=3D"Apple-converted-space"> class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; ">class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = >">Associate Professorstyle=3D"font-family: 'Gill Sans'; ">style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = >University
face=3D"GillSans-Light">style=3D"font-family: GillSans-Light; ">305 N. University Street | West = >Lafayette | IN 47907 | USA
class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans">class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = >'Gill Sans'; ">0, 255); font-family: 'Gill Sans'; ">Mobileclass=3D"Apple-style-span" face=3D"GillSans-Light">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; ">class=3D"Apple-converted-space"> +1 765 427 = >5484
face=3D"GillSans-Light">
class=3D"khtml-block-placeholder">
>

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline">

class=3D"Apple-interchange-newline"> >
>
On Aug 22, 2012, at 10:24 AM, Jay K <href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = >wrote:

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">why this loop/multipass stuff


So people = >understand where I'm going here -- and Tony asked. I had already written = >this up..



1) in the past working on m3cc/parse.c, I found = >that "declare"s
weren't always "topologically = >sorted".


That is, like, I might see:

given TYPE T1 =3D = >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = >seen type T1 declared before T2, or not.
I'm not sure if it was = >record/fields or other relationships.
I just remember that building = >up a map of typeid to data, and
looking up it when needed, didn't = >always work, with one pass.


So what I did is that I was = >willing to "skip" declares
that referred to typeids I hadn't seen = >yet, and then loop
again, hoping for progress at each loop (fewer = >skips).

Was this all in = >aid of building up proper type descriptors?

type=3D"cite">
font-family: Calibri; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = >the start" of a procedure. I'm not sure if I'm generating
C or C++, = >but if I'm generating C, it helps to have declare_local/temp
right = >after begin_procedure/block. Multiple passes let me make that
so, = >without changing m3front.
(Upon further looking..it seems this is = >mostly done right, but
not for "try" blocks; maybe reasonable to = >change m3front here.).


Now, generating C++ is an option, and = >I'm ignoring all declares
currently anyway, so these don't force the = >matter, but:


3) nested functions.
If a function contains = >any nested functions, I want to put all of
its locals in a local = >struct, and use the address of that struct
as the "static link". Or = >at least the locals that are referenced
by nested = >functions.

Preferably these = >only, or else you will cause all locals to be in memory instead of = >registers.

style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = >">
As well, m3front contains options as to
the order = >of outputing functions. It uses one order for m3back and
a different = >order for m3cc. I wouldn't mind if M3C.m3 worked = >either
way.


There are possibly other benefits that could = >be derived from multiple passes.
Nothing offhand = >though.


 - = >Jay

= > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- From jay.krell at cornell.edu Wed Aug 22 16:56:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Wed, 22 Aug 2012 14:56:31 +0000 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: <20120822144109.43DA71A2096@async.async.caltech.edu> References: , <20120822144109.43DA71A2096@async.async.caltech.edu> Message-ID: I'm implementing M3CG_Ops.m3. I don't trust m3tk -- it isn't the compiler. I'm somewhat familiar with m3cg. Given another M3CG, I know how to plug in to cm3, config files, etc. (e.g. add a new target, AMD64_NT, that will use this. :) ) On the other hand...I have a little more insight this morning. Variables have an "uplevel" parameter -- are they accessed from nested functions. I should be able to link variables to their containing procedure, and therefore its level. So "load(var)" I guess can be: text := "" for i := current_level to var.proc.level do (* typically an empty loop *) text := text & "_static_link->" end; if uplevel AND current_level = var.proc.level (* relatively rare *) text := text & "_locals."; end; text := text & var.name; or somesuch. I think I should get back to M3C with the assumption I don't need multiple passes...fil in more pieces..see where I get... - Jay > To: jay.krell at cornell.edu > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] why this loop/multipass stuff > Date: Wed, 22 Aug 2012 07:41:09 -0700 > From: mika at async.caltech.edu > > Jay I've been following what you've been writing with one eye. > > But I'm not clear exactly what framework you're fitting into. Are you > processing the output of m3front? > > There's always m3tk. Since it is claimed it was used to write a compiler > it ought to be able to process MODULEs but I've only ever used it to > process INTERFACEs. What's the story about that? > > Mika > > Tony Hosking writes: > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/plain; > > charset=iso-8859-1 > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > > > >On Aug 22, 2012, at 10:24 AM, Jay K wrote: > > > >> why this loop/multipass stuff > >>=20 > >>=20 > >> So people understand where I'm going here -- and Tony asked. I had = > >already written this up.. > >>=20 > >>=20 > >>=20 > >> 1) in the past working on m3cc/parse.c, I found that "declare"s > >> weren't always "topologically sorted". > >>=20 > >>=20 > >> That is, like, I might see: > >>=20 > >> given TYPE T1 =3D INTEGER; > >> TYPE T2 =3D RECORD =3D a:T1; END; > >>=20 > >>=20 > >> I might have seen type T1 declared before T2, or not. > >> I'm not sure if it was record/fields or other relationships. > >> I just remember that building up a map of typeid to data, and > >> looking up it when needed, didn't always work, with one pass. > >>=20 > >>=20 > >> So what I did is that I was willing to "skip" declares > >> that referred to typeids I hadn't seen yet, and then loop > >> again, hoping for progress at each loop (fewer skips). > > > >Was this all in aid of building up proper type descriptors? > > > >> 2) It looks like declare_local/temp don't all come in > >> "at the start" of a procedure. I'm not sure if I'm generating > >> C or C++, but if I'm generating C, it helps to have declare_local/temp > >> right after begin_procedure/block. Multiple passes let me make that > >> so, without changing m3front. > >> (Upon further looking..it seems this is mostly done right, but > >> not for "try" blocks; maybe reasonable to change m3front here.). > >>=20 > >>=20 > >> Now, generating C++ is an option, and I'm ignoring all declares > >> currently anyway, so these don't force the matter, but: > >>=20 > >>=20 > >> 3) nested functions. > >> If a function contains any nested functions, I want to put all of > >> its locals in a local struct, and use the address of that struct > >> as the "static link". Or at least the locals that are referenced > >> by nested functions. > > > >Preferably these only, or else you will cause all locals to be in memory = > >instead of registers. > > > >> As well, m3front contains options as to > >> the order of outputing functions. It uses one order for m3back and > >> a different order for m3cc. I wouldn't mind if M3C.m3 worked either > >> way. > >>=20 > >>=20 > >> There are possibly other benefits that could be derived from multiple = > >passes. > >> Nothing offhand though. > >>=20 > >>=20 > >> - Jay > > > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/html; > > charset=iso-8859-1 > > > > >charset=3Diso-8859-1"> >style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; ">
> > >color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-border-horizontal-spacing: 0px; = > >-webkit-border-vertical-spacing: 0px; = > >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; font-size: medium; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = > >rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = > >normal; font-variant: normal; font-weight: normal; letter-spacing: = > >normal; line-height: normal; orphans: 2; text-indent: 0px; = > >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = > >auto; -webkit-text-stroke-width: 0px; ">
>break-word; -webkit-nbsp-mode: space; -webkit-line-break: = > >after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; ">
>style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = > >-webkit-line-break: after-white-space; "> >style=3D"border-collapse: separate; -webkit-border-horizontal-spacing: = > >0px; -webkit-border-vertical-spacing: 0px; color: rgb(0, 0, 0); = > >font-family: Helvetica; font-size: 12px; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; -webkit-text-decorations-in-effect: none; = > >text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; = > >orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; "> >class=3D"Apple-style-span" style=3D"border-collapse: separate; = > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = > >0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = > >font-style: normal; font-variant: normal; font-weight: normal; = > >letter-spacing: normal; line-height: normal; = > >-webkit-text-decorations-in-effect: none; text-indent: 0px; = > >-webkit-text-size-adjust: auto; text-transform: none; orphans: 2; = > >white-space: normal; widows: 2; word-spacing: 0px; ">
>class=3D"Apple-style-span" color=3D"#0000FF"> >class=3D"Apple-style-span" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Antony = > >Hosking >face=3D"Gill Sans"> >'Gill Sans'; "> >'Gill Sans'; "> | >class=3D"Apple-converted-space">  >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; "> >class=3D"Apple-style-span" style=3D"font-family: 'Gill Sans'; = > >">Associate Professor >style=3D"font-family: 'Gill Sans'; "> >style=3D"font-family: 'Gill Sans'; "> | Computer Science | Purdue = > >University
> face=3D"GillSans-Light"> >style=3D"font-family: GillSans-Light; ">305 N. University Street | West = > >Lafayette | IN 47907 | USA
>class=3D"Apple-style-span" color=3D"#0000FF" face=3D"Gill Sans"> >class=3D"Apple-style-span" style=3D"color: rgb(0, 0, 255); font-family: = > >'Gill Sans'; "> >0, 255); font-family: 'Gill Sans'; ">Mobile >class=3D"Apple-style-span" face=3D"GillSans-Light"> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-style-span" style=3D"font-family: GillSans-Light; "> >class=3D"Apple-converted-space"> +1 765 427 = > >5484
>face=3D"GillSans-Light">
>class=3D"khtml-block-placeholder">
>>

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline">

>class=3D"Apple-interchange-newline"> > >
> >
On Aug 22, 2012, at 10:24 AM, Jay K < >href=3D"mailto:jay.krell at cornell.edu">jay.krell at cornell.edu> = > >wrote:

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">why this loop/multipass stuff


So people = > >understand where I'm going here -- and Tony asked. I had already written = > >this up..



1) in the past working on m3cc/parse.c, I found = > >that "declare"s
weren't always "topologically = > >sorted".


That is, like, I might see:

given TYPE T1 =3D = > >INTEGER;
TYPE T2 =3D RECORD =3D a:T1; END;


I might have = > >seen type T1 declared before T2, or not.
I'm not sure if it was = > >record/fields or other relationships.
I just remember that building = > >up a map of typeid to data, and
looking up it when needed, didn't = > >always work, with one pass.


So what I did is that I was = > >willing to "skip" declares
that referred to typeids I hadn't seen = > >yet, and then loop
again, hoping for progress at each loop (fewer = > >skips).

Was this all in = > >aid of building up proper type descriptors?

>type=3D"cite">
>font-family: Calibri; font-style: normal; font-variant: normal; = > >font-weight: normal; letter-spacing: normal; line-height: normal; = > >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = > >none; white-space: normal; widows: 2; word-spacing: 0px; = > >-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
>dir=3D"ltr">2) It looks like declare_local/temp don't all come in
"at = > >the start" of a procedure. I'm not sure if I'm generating
C or C++, = > >but if I'm generating C, it helps to have declare_local/temp
right = > >after begin_procedure/block. Multiple passes let me make that
so, = > >without changing m3front.
(Upon further looking..it seems this is = > >mostly done right, but
not for "try" blocks; maybe reasonable to = > >change m3front here.).


Now, generating C++ is an option, and = > >I'm ignoring all declares
currently anyway, so these don't force the = > >matter, but:


3) nested functions.
If a function contains = > >any nested functions, I want to put all of
its locals in a local = > >struct, and use the address of that struct
as the "static link". Or = > >at least the locals that are referenced
by nested = > >functions.

Preferably these = > >only, or else you will cause all locals to be in memory instead of = > >registers.

>style=3D"font-size: 12pt; font-family: Calibri; font-style: normal; = > >font-variant: normal; font-weight: normal; letter-spacing: normal; = > >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = > >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = > >0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = > >">
As well, m3front contains options as to
the order = > >of outputing functions. It uses one order for m3back and
a different = > >order for m3cc. I wouldn't mind if M3C.m3 worked = > >either
way.


There are possibly other benefits that could = > >be derived from multiple passes.
Nothing offhand = > >though.


 - = > >Jay

= > > > >--Apple-Mail=_ED26DAAF-73B8-4147-B1C9-82D48C3720BF-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 17:57:24 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 16:57:24 +0100 (BST) Subject: [M3devel] Fw: SW Bug #1080 In-Reply-To: <20120822092850.cef2e188.wagner@elego.de> Message-ID: <1345651044.13430.YahooMailClassic@web29705.mail.ird.yahoo.com> Hi all: I'm on the process of booting a cm3-5.8.6-REL but in the second phase bootstrap process (NOPTHREAD) I have broke the compiler with this message (before I have fixed 2 quake thread scripts: m3-libs/m3core/src/thread/m3makefile:13 drop not clause m3-libs/m3core/src/thread.quake:87 same as above and add at line 90: else ??? PosixKernelThreads = FALSE ??? KernelThreads = FALSE ) : root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts# ./do-pkg.sh buildship m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 /home/test/code/cm3-5.8.6-REL/scripts/pkgmap.sh -c "/usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' " m3core libm3 m3middle m3linker m3front sysutils m3quake m3objfile m3back cm3 === package /home/test/code/cm3-5.8.6-REL/m3-libs/m3core === ?+++ /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? +++ *** *** runtime error: ***??? Segmentation violation - possible attempt to dereference NIL ***??? pc = 0x8295954 = NoteStackLocations + 0x34 in ../src/runtime/common/RTCollector.m3 *** Aborted ?*** execution of /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -build -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS? && /usr/local/cm3/pkg/cm3/LINUXLIBC6/cm3 -ship $RARGS -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/home/test/code/cm3-5.8.6-REL' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11'? failed *** root at fbd-desktop:/home/test/code/cm3-5.8.6-REL/scripts#??? Antony, you have been busy but can you give us a clue on what to fix if so in m3core/src/runtime/common/RTCollector.m3 Thanks in advance --- El mi?, 22/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] Fw: SW Bug #1080 Para: m3devel at elegosoft.com CC: rob.zietlow at tdstelecom.com Fecha: mi?rcoles, 22 de agosto, 2012 02:28 I'm busy with a customer meeting all day; hopefully somebody from the devel list can take a look... Begin forwarded message: Date: Tue, 21 Aug 2012 17:25:54 +0000 From: "Zietlow, Rob" To: "'m3-support at elego.de'" Subject: SW Bug #1080 Hi support folks, I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo I have grabbed all of the sources for 5.8.6-REL.? I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship???when I attempt to run cvsup right now it hangs.? When I connect to the cvsupd it also hangs.???I have strace access from the PIDS if needed.? One of the threads I came across commented about building with userthreads.? I could not find that in the documentation. Any assistance would be helpful It is an RHEL 5u8 x86_64 system.? I can happily answer any questions you may have. Thanks Rob -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 22 19:37:12 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 22 Aug 2012 18:37:12 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822110048.903811A207D@async.async.caltech.edu> Message-ID: <1345657032.70947.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: in fact why not 'M3 as target language' is not the chicken and egg problem here, who needs C,? LLVM, stuff like that (we eat our own stuff and that's all), why not M3 in AST and that's all, like JSON la Olivetti. Olivetti tried to do that to compile and describe each node in the AST as an Object, we should explore the idea. http://lostechies.com/evanhoff/2008/01/08/language-roots/ Besides look for the 7th paragraph (I admire every day DEC-SRC and Modula-3 ever more) Thanks in advance --- El mi?, 22/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] higher level m3cg? Para: "Dirk Muysers" CC: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 06:00 In fairness, doesn't "C as target language" work OK for Pascal (p2c) and Fortran 77 (f2c)? ? ? Mika "Dirk Muysers" writes: >This is a multi-part message in MIME format. > >------=_NextPart_000_0018_01CD7F9F.8021D180 >Content-Type: text/plain; >??? charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >*** A warning *** >Norman Ramsey's opinion (in stackoverflow) on possible compiler = >backends: > >Code generation is my business :-) > >Comments on a few options: > >? a.. CLR:=20 > >? ? a.. Pro: industrial support=20 >? ? b.. Con: you have to buy into their type system pretty much = >completely; depending on what you want to do with types, this may not = >matter=20 >? ? c.. Con: Only Windows platform is really prime-time quality >? b.. LLVM: > >? ? a.. Pro: enthusiastic user community with charismatic leader=20 >? ? b.. Pro: serious backing from Apple=20 >? ? c.. Pro: many interesting performance improvements=20 >? ? d.. Con: somewhat complex interface=20 >? ? e.. Con: history of holes in the engineering; as LLVM matures expect = >the holes in the engineering to be plugged by adding to the complexity = >of the interface >? c.. C-- > >? ? a.. Pro: target is an actual written language, not an API; you can = >easily inspect, debug, and edit your C-- code=20 >? ? b.. Pro: design is reasonably mature and reasonably clean=20 >? ? c.. Pro: supports accurate garbage collection=20 >? ? d.. Pro: most users report it is very easy to use=20 >? ? e.. Con: very small development team=20 >? ? f.. Con: as of early 2009, supports only three hardware platforms = >(x86, PPC, ARM)=20 >? ? g.. Con: does not ship with a garbage collector=20 >? ? h.. Con: project has no future >? d.. C as target language > >? ? a.. Pro: looks easy=20 >? ? b.. Con: nearly impossible to get decent performance=20 >? ? c.. Con: will drive you nuts in the long run; ask the long line of = >people who have tried to compile Haskell, ML, Modula-3, Scheme and more = >using this technique. At some point every one of these people gave up = >and built their own native code generator. >Summary: anything except C is a reasonable choice. For the best = >combination of flexibility, quality, and expected longevity, I'd = >probably recommend LLVM. > >Full disclosure: I am affiliated with the C-- project. > >=20 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Wed Aug 22 20:11:06 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:11:06 -0400 Subject: [M3devel] why this loop/multipass stuff In-Reply-To: References: Message-ID: <20120822181106.GA14178@topoi.pooq.com> On Wed, Aug 22, 2012 at 02:24:42PM +0000, Jay K wrote: > why this loop/multipass stuff > > > So people understand where I'm going here -- and Tony asked. I had already written this up.. > > > > 1) in the past working on m3cc/parse.c, I found that "declare"s > weren't always "topologically sorted". > > > That is, like, I might see: > > given TYPE T1 = INTEGER; > TYPE T2 = RECORD = a:T1; END; > > > I might have seen type T1 declared before T2, or not. > I'm not sure if it was record/fields or other relationships. > I just remember that building up a map of typeid to data, and > looking up it when needed, didn't always work, with one pass. > > > So what I did is that I was willing to "skip" declares > that referred to typeids I hadn't seen yet, and then loop > again, hoping for progress at each loop (fewer skips). Does that mean you have a table of typeid's and their meanings in RAM during your multiple passes? If so, you might as well read them all in at once, and then link them together in RAM afterward. Or are you doing something sneakier? Or are you so short on RAM that thie is impractical? And for recorcds that are used by having pointers to them, you can just emit 'struct foo;' near the beginning of the generated C file. The topological sorting can be restricted to types that actually contain others, rather then pointing to them. This would reduce the amount of topological sorting you need to do. In fact, if you have all your types in RAM, you can do the topological sort *while* you are writing the types out. Just look through a type before emitting it and recurively emit any component types first. Mark each type when you emit it so you don't emit it twice. > > > 2) It looks like declare_local/temp don't all come in > "at the start" of a procedure. I'm not sure if I'm generating > C or C++, but if I'm generating C, it helps to have declare_local/temp > right after begin_procedure/block. Multiple passes let me make that > so, without changing m3front. > (Upon further looking..it seems this is mostly done right, but > not for "try" blocks; maybe reasonable to change m3front here.). How are you intending to make all these available for garbage collection? You might be interested in looking at Gambit/C, a Scheme compiler that compiles Schene to C and seems to do a fairly good job of it. > > > Now, generating C++ is an option, and I'm ignoring all declares > currently anyway, so these don't force the matter, but: > > > 3) nested functions. > If a function contains any nested functions, I want to put all of > its locals in a local struct, and use the address of that struct > as the "static link". Or at least the locals that are referenced > by nested functions. As well, m3front contains options as to > the order of outputing functions. It uses one order for m3back and > a different order for m3cc. I wouldn't mind if M3C.m3 worked either > way. I wanted to do this in my Algol 68 compiler when I was still considering using the LLVM API, but placing temporaries and the like into the struct too so as to make it easier to form a struct descriptor for it for the garbage collector to ues. But that meant I'd be adding fields to it while I was gernerating the code that used it, and LLVM objected to it. Ecen though its whole API is oriented to putting pieces of the parse tree in piecemeal, jumping all arounf the thing if necessary, it still demanded that types be completely defined before you could use them for anything much. In particular, they had to be completely defined before you could use the API to generate parse tree that used the field names of the structure. This pretty well made the LLVM API significantly less attractive for this project. > > > There are possibly other benefits that could be derived from multiple passes. > Nothing offhand though. > > > - Jay > From hendrik at topoi.pooq.com Wed Aug 22 20:38:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 22 Aug 2012 14:38:52 -0400 Subject: [M3devel] higher level m3cg? In-Reply-To: References: Message-ID: <20120822183852.GB14178@topoi.pooq.com> On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > > a.. CLR: > > a.. Pro: industrial support > b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter > c.. Con: Only Windows platform is really prime-time quality > b.. LLVM: > > a.. Pro: enthusiastic user community with charismatic leader > b.. Pro: serious backing from Apple > c.. Pro: many interesting performance improvements > d.. Con: somewhat complex interface > e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead. Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to. Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again. If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish. This interfered with building run-time data structures whose full contents would not be known until compilation was complete. > c.. C-- > > a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code > b.. Pro: design is reasonably mature and reasonably clean > c.. Pro: supports accurate garbage collection > d.. Pro: most users report it is very easy to use > e.. Con: very small development team > f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) > g.. Con: does not ship with a garbage collector > h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API. Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward. Of course this could be done with LLVM too. About the garbage collector. The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution. The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. > d.. C as target language > > a.. Pro: looks easy > b.. Con: nearly impossible to get decent performance > c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > > > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. > It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: > enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; > typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > > - Jay From dabenavidesd at yahoo.es Thu Aug 23 02:51:46 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 01:51:46 +0100 (BST) Subject: [M3devel] higher level m3cg? In-Reply-To: <20120822183852.GB14178@topoi.pooq.com> Message-ID: <1345683106.74499.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I was reading an article and one a good consequence of Using a type safe like CLR, C-- would be be able to detect deadlocks with type inference. http://arxiv.org/pdarf/1002.0942.pdf So it can be good to do that for safety reasons, but at the cost of type inference time, which I think you wouldn't like too much for JIT compilers. Thanks in advance --- El mi?, 22/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] higher level m3cg? Para: m3devel at elegosoft.com Fecha: mi?rcoles, 22 de agosto, 2012 13:38 On Tue, Aug 21, 2012 at 01:18:48PM +0200, Dirk Muysers wrote: > *** A warning *** > Norman Ramsey's opinion (in stackoverflow) on possible compiler backends: > > Code generation is my business :-) > > Comments on a few options: > >???a.. CLR: > >? ???a.. Pro: industrial support >? ???b.. Con: you have to buy into their type system pretty much completely; depending on what you want to do with types, this may not matter >? ???c.. Con: Only Windows platform is really prime-time quality >???b.. LLVM: > >? ???a.. Pro: enthusiastic user community with charismatic leader >? ???b.. Pro: serious backing from Apple >? ???c.. Pro: many interesting performance improvements >? ???d.. Con: somewhat complex interface >? ???e.. Con: history of holes in the engineering; as LLVM matures expect the holes in the engineering to be plugged by adding to the complexity of the interface When I was investigating using LLVM for Algol 68, I ran into one problem that persuaded me to use C-- instead.? Pointer arithmetic is C's pointer arithmetic; that is adding an integer to a pointer automatically scales the integer by the size of the thing pointed to.? Now arrays in C++ have the property that the stride -- the offset from each element to the next -- in not neccarily a multiple of the item pointed to, so using LLVm's pointer arithmetic is pretty well excluded -- without kludhes like casting everything to void* and then casting it back again.? If LLVM does any kind of type-based optimization, this will effectively disable it. I prefer to use things the way they were intended to be used, bot to abuse them and hope things still work. Of course the same objections apply to C and C++. There was also a problem with having to completely define every structure before you can even put any of its field names in the parse tree you can in principle generate in any order you wish.? This interfered with building run-time data structures whose full contents would not be known until compilation was complete. ? >???c.. C-- > >? ???a.. Pro: target is an actual written language, not an API; you can easily inspect, debug, and edit your C-- code >? ???b.. Pro: design is reasonably mature and reasonably clean >? ???c.. Pro: supports accurate garbage collection >? ???d.. Pro: most users report it is very easy to use >? ???e.. Con: very small development team >? ???f.. Con: as of early 2009, supports only three hardware platforms (x86, PPC, ARM) >? ???g.. Con: does not ship with a garbage collector >? ???h.. Con: project has no future C-- did allow unscaled pointer arithmetic. C-- did not have any API calling order restrictions, because it didn't have an API.? Any define-before-use restrictions could be fudged by generating output as several files and then concatenating them afterward.? Of course this could be done with LLVM too. About the garbage collector.? The world needs a robust garbage collector supports multicore systems on which collection is concurrent with execution.? The necessary synchronisation will cause performance loss, but there are applications where chunky garbage-colections delays are unaccepatable, bot general slowness is OK. Of course such a collector could have compile-time (or maybe run-time) options whether to support expensive features for applications where they are not relevant. >???d.. C as target language > >? ???a.. Pro: looks easy >? ???b.. Con: nearly impossible to get decent performance >? ???c.. Con: will drive you nuts in the long run; ask the long line of people who have tried to compile Haskell, ML, Modula-3, Scheme and more using this technique. At some point every one of these people gave up and built their own native code generator. > Summary: anything except C is a reasonable choice. For the best combination of flexibility, quality, and expected longevity, I'd probably recommend LLVM. > > Full disclosure: I am affiliated with the C-- project. Yay! > >? > > > From: Jay K > Sent: Thursday, August 16, 2012 4:21 PM > To: m3devel > Subject: [M3devel] higher level m3cg? > > > Should m3cg provide enough information for a backend to generate idiomatic C? > (What is idiomatic C? e.g. I'm ignoring loop constructs and exception handlinh..) > > > Should we make it so? > > > Or be pragmatic and see if anyone gets to that point? > > > But, look at this another way. > Let's say we are keeping the gcc backend. > > > Isn't it reasonable to have a better experience with stock gdb? > > > What should m3cg look like then? > > > Matching up m3front to gcc turns out to be "wierd". > As does having a backend generate "C". > > > In particular, "wierd" because there is a "level mismatch". > > > m3cg presents a fairly low level view of the program. >???It does layout. Global variables are stuffed into what you might call a "struct", with > no assigned field names. Field references are done by adding to addresses and casting. > > > Too low level to provide a "good" gcc tree representation or to generate "normal" C. > > > One might be able to, by somewhat extraordinary means, make due. > That is, specifically one could deduce field references from > offsets/sizes. But maybe it is reasonable for load/store > to include fields? Maybe in addition to what it provides? > > > As well, it appears to me, that > > > given TYPE Enum = {One, Two, Three}; > > the m3cg is like: > > declare enum typeidblah > declare enum_elt One > declare enum_elt Two > declare enum_elt Three > declare_typename typeidblah Enum > > > One kind of instead wants more like: > > > declare enum typeidblah Enum > declare enum_elt One => rename it Enum_One > declare enum_elt Two "" > declare enum_elt Three "" > > > However I understand that {One, Two, Three} exists > as anonymous type independent of the name "Enum". > > > One could just as well have: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = {One, Two, Three}; > > > Enum1 and Enum2 probably have the same typeid, and are just > two typenames for the same type. > > > likewise: > given TYPE Enum1 = {One, Two, Three}; > given TYPE Enum2 = Enum1; > > > but, pragmatically, in the interest of generating better C, > can we pass a name along with declare_enum? > > I ask somewhat rhetorically. I realize there is the answer: >???enum Mtypeid { Mtypeid_One, Mtypeid_Two, Mtypeid_Three }; >???typedef enum Mtypeid Enum1; > > > Also, enum variables I believe end up as just UINT8, 16, or 32. > Loads of enum values I believe end up as just loads of integers. > Can we pass along optional enum names with declare_local/declare_param? > And optional enum names with load_int? > Or add a separate load_enum call? > > > Really, I understand that the current interface can be pressed to do > pretty adequate things. I can infer field references. The way enums work > isn't too bad. > > >? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 09:24:05 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 09:24:05 +0200 Subject: [M3devel] I know, I know... Message-ID: I know this will probably be very dense subject, but? What about: TSPacketHeader = <* ENDIAN = BIG*>RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) tErrInd, (* Transport Error Indicator *) pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) transPrio: BITS 1 FOR [0..1]; pid: BITS 13 FOR PID; transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) cc: BITS 4 FOR Nibble; END; Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: TSPacketHeader = RECORD sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) pidHi: BITS 5 FOR [16_00..16_1f]; transPrio: BITS 1 FOR [0..1]; pusi, tErrInd: BITS 1 FOR BOOLEAN; pidLo: BITS 8 FOR [16_00..16_ff]; cc: BITS 4 FOR Nibble; afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) END; And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. Thanks in advance.. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hosking at cs.purdue.edu Thu Aug 23 15:30:57 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 23 Aug 2012 09:30:57 -0400 Subject: [M3devel] I know, I know... In-Reply-To: References: Message-ID: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> PACKED? Sent from my iPad On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > I know this will probably be very dense subject, but? What about: > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > tErrInd, (* Transport Error Indicator *) > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > transPrio: BITS 1 FOR [0..1]; > pid: BITS 13 FOR PID; > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > cc: BITS 4 FOR Nibble; > END; > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > TSPacketHeader = RECORD > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > pidHi: BITS 5 FOR [16_00..16_1f]; > transPrio: BITS 1 FOR [0..1]; > pusi, > tErrInd: BITS 1 FOR BOOLEAN; > > pidLo: BITS 8 FOR [16_00..16_ff]; > > cc: BITS 4 FOR Nibble; > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > END; > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > Thanks in advance.. > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > From wagner at elego.de Thu Aug 23 16:24:03 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:24:03 +0200 Subject: [M3devel] CVSup hangs, was: Re: SW Bug #1080 In-Reply-To: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> References: <6924DB838EC56549BBAA755EA408BBDCBEFFD7@cmailbox1> Message-ID: <20120823162403.a2b0f83b.wagner@elego.de> On Wed, 22 Aug 2012 09:57:04 +0000 "Zietlow, Rob" wrote: > I have been looking through the bugs and I seem to be experiencing one that was listed in bug 1080. However I am also experiencing hangs when I run the cvsup command and pull from a local repo > > I have grabbed all of the sources for 5.8.6-REL. I used these and the scripts associated with it to build up ./do-cm3-std.sh build and ship when I attempt to run cvsup right now it hangs. When I connect to the cvsupd it also hangs. I have strace access from the PIDS if needed. One of the threads I came across commented about building with userthreads. I could not find that in the documentation. I think it might be helpful if you run the program in a debugger (gdb, m3gdb) and get a stack backtrace when it hangs, then post it to the list. Another approach might be to check if compilation with current sources will yield better results. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elego.de Thu Aug 23 16:33:19 2012 From: wagner at elego.de (Olaf Wagner) Date: Thu, 23 Aug 2012 16:33:19 +0200 Subject: [M3devel] recent and old bug reports Message-ID: <20120823163319.35c5bd77.wagner@elego.de> Is anybody out there with some time who could look into these issues: #1275 RTCollector problem when NOPTHREAD RT used libs 5.8.6 support wagner new 08/22/12 #1080 CVSUPD server hangs if used with -C option sys 5.8-RC3 sw-bug wagner new 12/29/09 #1148 CVSup stalls on several platforms misc 5.8.6 CM3 Release 5.9 sw-bug wagner new 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dragisha at m3w.org Thu Aug 23 17:25:28 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 17:25:28 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <3E9D81B8-0FC4-404A-A439-BEB21627F30E@m3w.org> packing, order, alignment. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 3:30 PM, Tony Hosking wrote: > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > >> I know this will probably be very dense subject, but? What about: >> >> TSPacketHeader = <* ENDIAN = BIG*>RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> tErrInd, (* Transport Error Indicator *) >> pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> transPrio: BITS 1 FOR [0..1]; >> pid: BITS 13 FOR PID; >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> cc: BITS 4 FOR Nibble; >> END; >> >> Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> >> Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> >> TSPacketHeader = RECORD >> sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> >> pidHi: BITS 5 FOR [16_00..16_1f]; >> transPrio: BITS 1 FOR [0..1]; >> pusi, >> tErrInd: BITS 1 FOR BOOLEAN; >> >> pidLo: BITS 8 FOR [16_00..16_ff]; >> >> cc: BITS 4 FOR Nibble; >> afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> END; >> >> And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> >> Thanks in advance.. >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 19:07:47 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:07:47 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741667.48136.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 23 19:08:03 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 18:08:03 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <20120823163319.35c5bd77.wagner@elego.de> Message-ID: <1345741683.90835.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:29:00 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:29:00 +0200 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> You mean, for system programming I have to use C? -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:26 PM, Jay K wrote: > How about you just use "bytes" or integers and helper functions like this: > > > typedef struct _BIT_FIELD { > UINT8 ByteOffset; > UINT8 BitOffset; > UINT8 BitCount; > } BIT_FIELD; > > > HRESULT > ExtractBitField(__in const BIT_FIELD* BitField, > __in const UINT8* Bytes, > __in size_t Size, > __out_opt UINT8* Value) > /*++ > Extract a bitfield from an array of bytes, as specified > by a byte offset, a bit offset within the first byte, and a bit count. > The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. > The bitfield can definitely cross byte boundaries. > --*/ > { > UINT8 const bitCount = BitField->BitCount; > UINT8 const bitOffset = BitField->BitOffset; > UINT8 const byteOffset = BitField->ByteOffset; > UINT8 const FF = (UINT8)~(UINT8)0; > BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); > UINT8 const secondBitOffset = (8 - bitOffset); > UINT8 localValue = { 0 }; > HRESULT error = { 0 }; > if (Value) > *Value = 0; > if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) > { > error = E_INVALIDARG; > goto Exit; > } > if ((byteOffset + twoBytes) >= Size) > { > error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); > goto Exit; > } > localValue = (Bytes[byteOffset] >> bitOffset); > if (twoBytes) > { > localValue &= ~(FF << secondBitOffset); > localValue |= (Bytes[byteOffset + 1] << secondBitOffset); > } > localValue &= ~(FF << bitCount); > error = 0; > if (Value) > *Value = localValue; > Exit: > return error; > } > > > arrays of bytes are really the only way to control the layout. > Look at how GNU binutils works, for example... > Bitfields in C don't yield predictable/portable layout either. > > > - Jay > > > From: hosking at cs.purdue.edu > > Date: Thu, 23 Aug 2012 09:30:57 -0400 > > To: dragisha at m3w.org > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] I know, I know... > > > > PACKED? > > > > Sent from my iPad > > > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > > > I know this will probably be very dense subject, but. What about: > > > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > tErrInd, (* Transport Error Indicator *) > > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > > transPrio: BITS 1 FOR [0..1]; > > > pid: BITS 13 FOR PID; > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > cc: BITS 4 FOR Nibble; > > > END; > > > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > > > TSPacketHeader = RECORD > > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > > transPrio: BITS 1 FOR [0..1]; > > > pusi, > > > tErrInd: BITS 1 FOR BOOLEAN; > > > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > > > cc: BITS 4 FOR Nibble; > > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > > END; > > > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > > > Thanks in advance.. > > > -- > > > Divided by a common language > > > > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Thu Aug 23 21:26:58 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 19:26:58 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> Message-ID: How about you just use "bytes" or integers and helper functions like this: typedef struct _BIT_FIELD { UINT8 ByteOffset; UINT8 BitOffset; UINT8 BitCount; } BIT_FIELD; HRESULT ExtractBitField(__in const BIT_FIELD* BitField, __in const UINT8* Bytes, __in size_t Size, __out_opt UINT8* Value) /*++ Extract a bitfield from an array of bytes, as specified by a byte offset, a bit offset within the first byte, and a bit count. The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. The bitfield can definitely cross byte boundaries. --*/ { UINT8 const bitCount = BitField->BitCount; UINT8 const bitOffset = BitField->BitOffset; UINT8 const byteOffset = BitField->ByteOffset; UINT8 const FF = (UINT8)~(UINT8)0; BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); UINT8 const secondBitOffset = (8 - bitOffset); UINT8 localValue = { 0 }; HRESULT error = { 0 }; if (Value) *Value = 0; if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) { error = E_INVALIDARG; goto Exit; } if ((byteOffset + twoBytes) >= Size) { error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); goto Exit; } localValue = (Bytes[byteOffset] >> bitOffset); if (twoBytes) { localValue &= ~(FF << secondBitOffset); localValue |= (Bytes[byteOffset + 1] << secondBitOffset); } localValue &= ~(FF << bitCount); error = 0; if (Value) *Value = localValue; Exit: return error; } arrays of bytes are really the only way to control the layout.Look at how GNU binutils works, for example...Bitfields in C don't yield predictable/portable layout either. - Jay > From: hosking at cs.purdue.edu > Date: Thu, 23 Aug 2012 09:30:57 -0400 > To: dragisha at m3w.org > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > PACKED? > > Sent from my iPad > > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: > > > I know this will probably be very dense subject, but. What about: > > > > TSPacketHeader = <* ENDIAN = BIG*>RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > tErrInd, (* Transport Error Indicator *) > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) > > transPrio: BITS 1 FOR [0..1]; > > pid: BITS 13 FOR PID; > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > cc: BITS 4 FOR Nibble; > > END; > > > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. > > > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: > > > > TSPacketHeader = RECORD > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) > > > > pidHi: BITS 5 FOR [16_00..16_1f]; > > transPrio: BITS 1 FOR [0..1]; > > pusi, > > tErrInd: BITS 1 FOR BOOLEAN; > > > > pidLo: BITS 8 FOR [16_00..16_ff]; > > > > cc: BITS 4 FOR Nibble; > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) > > END; > > > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. > > > > Thanks in advance.. > > -- > > Divided by a common language > > > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Thu Aug 23 21:42:51 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:42:51 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <25147774-5A4A-4793-921A-2C9101273773@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> Message-ID: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > You mean, for system programming I have to use C? > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:26 PM, Jay K wrote: > >> How about you just use "bytes" or integers and helper functions like this: >> >> >> typedef struct _BIT_FIELD { >> UINT8 ByteOffset; >> UINT8 BitOffset; >> UINT8 BitCount; >> } BIT_FIELD; >> >> >> HRESULT >> ExtractBitField(__in const BIT_FIELD* BitField, >> __in const UINT8* Bytes, >> __in size_t Size, >> __out_opt UINT8* Value) >> /*++ >> Extract a bitfield from an array of bytes, as specified >> by a byte offset, a bit offset within the first byte, and a bit count. >> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >> The bitfield can definitely cross byte boundaries. >> --*/ >> { >> UINT8 const bitCount = BitField->BitCount; >> UINT8 const bitOffset = BitField->BitOffset; >> UINT8 const byteOffset = BitField->ByteOffset; >> UINT8 const FF = (UINT8)~(UINT8)0; >> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >> UINT8 const secondBitOffset = (8 - bitOffset); >> UINT8 localValue = { 0 }; >> HRESULT error = { 0 }; >> if (Value) >> *Value = 0; >> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >> { >> error = E_INVALIDARG; >> goto Exit; >> } >> if ((byteOffset + twoBytes) >= Size) >> { >> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >> goto Exit; >> } >> localValue = (Bytes[byteOffset] >> bitOffset); >> if (twoBytes) >> { >> localValue &= ~(FF << secondBitOffset); >> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >> } >> localValue &= ~(FF << bitCount); >> error = 0; >> if (Value) >> *Value = localValue; >> Exit: >> return error; >> } >> >> >> arrays of bytes are really the only way to control the layout. >> Look at how GNU binutils works, for example... >> Bitfields in C don't yield predictable/portable layout either. >> >> >> - Jay >> >> > From: hosking at cs.purdue.edu >> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >> > To: dragisha at m3w.org >> > CC: m3devel at elegosoft.com >> > Subject: Re: [M3devel] I know, I know... >> > >> > PACKED? >> > >> > Sent from my iPad >> > >> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >> > >> > > I know this will probably be very dense subject, but. What about: >> > > >> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > tErrInd, (* Transport Error Indicator *) >> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >> > > transPrio: BITS 1 FOR [0..1]; >> > > pid: BITS 13 FOR PID; >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > cc: BITS 4 FOR Nibble; >> > > END; >> > > >> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >> > > >> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >> > > >> > > TSPacketHeader = RECORD >> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >> > > >> > > pidHi: BITS 5 FOR [16_00..16_1f]; >> > > transPrio: BITS 1 FOR [0..1]; >> > > pusi, >> > > tErrInd: BITS 1 FOR BOOLEAN; >> > > >> > > pidLo: BITS 8 FOR [16_00..16_ff]; >> > > >> > > cc: BITS 4 FOR Nibble; >> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >> > > END; >> > > >> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >> > > >> > > Thanks in advance.. >> > > -- >> > > Divided by a common language >> > > >> > > Dragi?a Duri? >> > > dragisha at m3w.org >> > > >> > > >> > > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Thu Aug 23 21:51:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 12:51:15 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: <20120823195115.80C121A2094@async.async.caltech.edu> Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" > > >--Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I have a friend, working for large software company and he just recently = >ported some message router from SPARC to Linux/Intel=85 Lots of "network = >order" data, same as with my current projects. Network, communication in = >general, network order is everywhere. > >Insisting on some god-given data ordering is a bit=85 What is nice word = >here? :) Outlandish? Outimeish? > >This is not first time I started discussion like this here. Every single = >time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >as I have shown in my example, with Modula-3 code. With a bit of effort = >I can make it almost-transparent (subfolder/platform) over various = >platforms. Of course, as I am developing a full product (not software to = >be run on arbitrary platform) I don't have to worry about too many = >platforms. One is enough here, but I still think Modula-3 can benefit, = >and a lot, if it supported explicit byte/bit ordering/packing/aligning = >pragmas. > >Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >we must write piece of software in C to be what? Compatible with API's = >all systems are compatible with without any hassle.=20 > >TIA >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From dragisha at m3w.org Thu Aug 23 21:59:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 23 Aug 2012 21:59:46 +0200 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like? GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas of your own choice (or comments, or auxiliary data > from somewhere) and then generates code using Word.T that can process > byte streams and return the RECORD you declared, in portable Modula-3. > I doubt it's very difficult. m3tk is cool! > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" >> >> >> --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I have a friend, working for large software company and he just recently = >> ported some message router from SPARC to Linux/Intel=85 Lots of "network = >> order" data, same as with my current projects. Network, communication in = >> general, network order is everywhere. >> >> Insisting on some god-given data ordering is a bit=85 What is nice word = >> here? :) Outlandish? Outimeish? >> >> This is not first time I started discussion like this here. Every single = >> time Jay explains to me how Modula-3 cannot handle it. I am handling it, = >> as I have shown in my example, with Modula-3 code. With a bit of effort = >> I can make it almost-transparent (subfolder/platform) over various = >> platforms. Of course, as I am developing a full product (not software to = >> be run on arbitrary platform) I don't have to worry about too many = >> platforms. One is enough here, but I still think Modula-3 can benefit, = >> and a lot, if it supported explicit byte/bit ordering/packing/aligning = >> pragmas. >> >> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = >> we must write piece of software in C to be what? Compatible with API's = >> all systems are compatible with without any hassle.=20 >> >> TIA >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dabenavidesd at yahoo.es Thu Aug 23 22:34:09 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 23 Aug 2012 21:34:09 +0100 (BST) Subject: [M3devel] recent and old bug reports In-Reply-To: <2BADF386-2CFB-43EE-B809-F965AE351CD2@gmail.com> Message-ID: <1345754049.2762.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: just If you care for me, I don't have another machine to make bootstrap archives from newer compilers (that I don't have), and I haven't been following the trunck since when this stuff started to happen with CVSup (I think I still have the "mist stable RC3 or so"), so technically and morally, I have a dilemma, would need a lottery to make compilers work in so a long time when upgrading the compiler (maybe not). Manually I don't feel it's possible either as I lack the knowledge to do that either, so that's why I needed bootstrap archives but nobody answered here so... Thanks in advance --- El jue, 23/8/12, Antony Hosking escribi?: De: Antony Hosking Asunto: Re: [M3devel] recent and old bug reports Para: "Daniel Alejandro Benavides D." CC: "m3devel at elegosoft.com" , "Olaf Wagner" Fecha: jueves, 23 de agosto, 2012 13:50 There are a number if bugs in the release that have since been fixed in the trunk. Best to upgrade from sources before trying to diagnose.? Sent from my iPhone On Aug 23, 2012, at 13:07, "Daniel Alejandro Benavides D." wrote: Hi all: Is anybody out there with legacy systems to try this out that can test those platforms against this bugs, if I'm correct the thread handling is the main issue here at the processor level, weakly defined semantics makes room for bad implementations on the kernel (so tickets won't be enough to count the same problem in any place, is a structural problem). Jay and Antony had already put lots of time to CVSup (I might give it a time if you will but I lack good HW and more knowledge) Jay, Antony what about this feature ticket? https://cm3-bugs.elegosoft.com/cm3/ticket/1029 Thanks in advance --- El jue, 23/8/12, Olaf Wagner escribi?: De: Olaf Wagner Asunto: [M3devel] recent and old bug reports Para: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 09:33 Is anybody out there with some time who could look into these issues: #1275 ??? RTCollector problem when NOPTHREAD RT used ??? libs ??? 5.8.6 ??? ??? support ??? wagner ??? new ??? 08/22/12 #1080 ??? CVSUPD server hangs if used with -C option ??? sys ??? 5.8-RC3 ??? ??? sw-bug ??? wagner ??? new ??? 12/29/09 #1148 ??? CVSup stalls on several platforms ??? misc ??? 5.8.6 ??? CM3 Release 5.9 ??? sw-bug ??? wagner ??? new ??? 08/27/10 https://cm3-bugs.elegosoft.com/cm3/ticket/1275 https://cm3-bugs.elegosoft.com/cm3/ticket/1080 https://cm3-bugs.elegosoft.com/cm3/ticket/1148 These have been recently reported or seem to be related. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com ? ? ? ? ? ? ???Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96? mobile: +49 177 2345 869? fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 00:12:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Aug 2012 22:12:48 +0000 Subject: [M3devel] I know, I know... In-Reply-To: References: , , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, Message-ID: I don't doubt that Modula-3 can handle it. And I don't doubt that C can handle it. But in both cases I find 1) the details beyond my attention span 2) definitely in the C case and very possibly in the Modula-3 case, not portable. You could learn the rules on one platform/compiler, only to produce code that doesn't work on others. Why bother learning them therefore? If you just want efficient storage in memory and don't care about the layout, then bitfields are ok. If you need to control the layout to interoperate with another system..even though bitfields are supposedly designed for this..I'm not sure I'd bother. C bitfields don't have portable layout. Do you suppose that Modula-3 ones do? Maybe there are in fact an obvious set of rules that all compilers follow? I don't know. Arrays of bytes have predictable layout. Look at how binutils reads/writes object files. It does like this: typedef struct elf_header_packed { char signature[4]; char architecture[4]; char offset_to_foo[4]; } elf_header_packed; typedef struct elf_header { long signature; long architecture; long offset_to_foo; } elf_header; and uses some pack/unpack functions to convert. They are solving a slightly different problem. They don't have bitfields necessarily to contend with, but just packing/size of integers that have a number of bits that are power of 2 (8, 16, 32, 64). - Jay From: dragisha at m3w.org Date: Thu, 23 Aug 2012 21:59:46 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] I know, I know... I have RTInfo module we developed years ago to parse .M3WEB. Good enough? :) But, not a solution, it is easier to write as I wrote in example I used to illustrate problem. GCC not obeying packing instructions is like. GCC not working in lot of cases. For some reason, I don't believe in it. Hint, gm2 is GCC based compiler. And supports very precise packing/aligning rules. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 (you can just use Word for that). But I think the point was that a compiler back-end might not necessarily obey your packing instructions? Here's what I think you should do... it would be super useful, both to you and other people in the future. And completely independent of compilers. Write a something-or-other that uses m3tk to read your RECORD type spec, plus pragmas of your own choice (or comments, or auxiliary data from somewhere) and then generates code using Word.T that can process byte streams and return the RECORD you declared, in portable Modula-3. I doubt it's very difficult. m3tk is cool! Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: --Apple-Mail=_8871A9FB-3130-4B36-8524-F6549A5CEC9E Content-Type: multipart/alternative; boundary="Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122" --Apple-Mail=_2314A877-555E-4BDE-8112-C875A2AE5122 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1250 I have a friend, working for large software company and he just recently = ported some message router from SPARC to Linux/Intel=85 Lots of "network = order" data, same as with my current projects. Network, communication in = general, network order is everywhere. Insisting on some god-given data ordering is a bit=85 What is nice word = here? :) Outlandish? Outimeish? This is not first time I started discussion like this here. Every single = time Jay explains to me how Modula-3 cannot handle it. I am handling it, = as I have shown in my example, with Modula-3 code. With a bit of effort = I can make it almost-transparent (subfolder/platform) over various = platforms. Of course, as I am developing a full product (not software to = be run on arbitrary platform) I don't have to worry about too many = platforms. One is enough here, but I still think Modula-3 can benefit, = and a lot, if it supported explicit byte/bit ordering/packing/aligning = pragmas. Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so = we must write piece of software in C to be what? Compatible with API's = all systems are compatible with without any hassle.=20 TIA -- Divided by a common language Dragi=9Aa Duri=E6 dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Aug 24 01:07:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 23 Aug 2012 16:07:44 -0700 Subject: [M3devel] I know, I know... In-Reply-To: References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120823230744.509A61A2094@async.async.caltech.edu> Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before. If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline. Then I'd write a template file that allows me to specify said Scheme file in m3makefile. The advantage of this approach is that it doesn't rely on any compiler internals or special output files. And it's at least as powerful as any pragmas you might want to add. Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways). I only define two tables, you can imagine what the rest looks like. It's for a real-time financial trading application, if you were wondering. The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL. The code generator is only 1,197 lines of Scheme and that is including liberal documentation. (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.) The whole thing is added to the program by adding a single line to m3makefile. Mika (define aux-mm-ordr-ctrl (make-table "aux_mm_ordr_ctrl" 'client (make-field "symbol" 'varchar 'not-null) (make-index (list "symbol")) (make-field "type" 'varchar 'not-null) (make-field "expiry" 'varchar 'not-null) (make-field "exchange" 'varchar 'not-null) (make-field "currency" 'varchar 'not-null) (make-field "action" 'varchar 'not-null) (make-field "quantity" 'double-precision 'not-null) (make-field "order_type" 'varchar 'not-null) (make-field "benchmark_price" 'double-precision) (make-field "time_allowance" 'double-precision) (make-field "model_name" 'varchar) (make-index (list "model_name")) (make-field "id" 'integer) )) (define clean (make-table "clean" 'server (make-field "tabl" 'varchar 'not-null) (make-field "rowid" 'integer 'not-null) (make-field "client" 'varchar 'not-null) (make-index (list "client")) (make-index (list "tabl")) (make-index (list "tabl" "rowid")) (make-index (list "client" "tabl")))) (define gcoms (make-database "gcoms" clean code model instrument strategy model-data model-code ordr ordr_status output eror fill aux-instrument aux-ordr aux-ordr-params aux-market-segment aux-last-prices aux-mm-ordr-ctrl aux-mm-associations )) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; > boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future. And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult. m3tk is cool! >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre"> = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- From jay.krell at cornell.edu Fri Aug 24 03:54:29 2012 From: jay.krell at cornell.edu (Jay) Date: Thu, 23 Aug 2012 18:54:29 -0700 Subject: [M3devel] I know, I know... In-Reply-To: <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> Message-ID: Look at the code in m3core or libm3 that picks apart floats? The unsafe/loophole part isn't relevant. - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 23, 2012, at 12:42 PM, Dragi?a Duri? wrote: > I have a friend, working for large software company and he just recently ported some message router from SPARC to Linux/Intel? Lots of "network order" data, same as with my current projects. Network, communication in general, network order is everywhere. > > Insisting on some god-given data ordering is a bit? What is nice word here? :) Outlandish? Outimeish? > > This is not first time I started discussion like this here. Every single time Jay explains to me how Modula-3 cannot handle it. I am handling it, as I have shown in my example, with Modula-3 code. With a bit of effort I can make it almost-transparent (subfolder/platform) over various platforms. Of course, as I am developing a full product (not software to be run on arbitrary platform) I don't have to worry about too many platforms. One is enough here, but I still think Modula-3 can benefit, and a lot, if it supported explicit byte/bit ordering/packing/aligning pragmas. > > Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. It is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit so we must write piece of software in C to be what? Compatible with API's all systems are compatible with without any hassle. > > TIA > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 23, 2012, at 9:29 PM, Dragi?a Duri? wrote: > >> You mean, for system programming I have to use C? >> >> -- >> Divided by a common language >> >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> >> >> On Aug 23, 2012, at 9:26 PM, Jay K wrote: >> >>> How about you just use "bytes" or integers and helper functions like this: >>> >>> >>> typedef struct _BIT_FIELD { >>> UINT8 ByteOffset; >>> UINT8 BitOffset; >>> UINT8 BitCount; >>> } BIT_FIELD; >>> >>> >>> HRESULT >>> ExtractBitField(__in const BIT_FIELD* BitField, >>> __in const UINT8* Bytes, >>> __in size_t Size, >>> __out_opt UINT8* Value) >>> /*++ >>> Extract a bitfield from an array of bytes, as specified >>> by a byte offset, a bit offset within the first byte, and a bit count. >>> The bitfield is presumed to be no more than 8 bits -- however that could be easily fixed. >>> The bitfield can definitely cross byte boundaries. >>> --*/ >>> { >>> UINT8 const bitCount = BitField->BitCount; >>> UINT8 const bitOffset = BitField->BitOffset; >>> UINT8 const byteOffset = BitField->ByteOffset; >>> UINT8 const FF = (UINT8)~(UINT8)0; >>> BOOL8 const twoBytes = ((bitOffset + bitCount) > 8); >>> UINT8 const secondBitOffset = (8 - bitOffset); >>> UINT8 localValue = { 0 }; >>> HRESULT error = { 0 }; >>> if (Value) >>> *Value = 0; >>> if ((bitCount > 8) || (bitOffset > 7) || (bitCount == 0)) >>> { >>> error = E_INVALIDARG; >>> goto Exit; >>> } >>> if ((byteOffset + twoBytes) >= Size) >>> { >>> error = HRESULT_FROM_WIN32(ERROR_BUFFER_OVERFLOW); >>> goto Exit; >>> } >>> localValue = (Bytes[byteOffset] >> bitOffset); >>> if (twoBytes) >>> { >>> localValue &= ~(FF << secondBitOffset); >>> localValue |= (Bytes[byteOffset + 1] << secondBitOffset); >>> } >>> localValue &= ~(FF << bitCount); >>> error = 0; >>> if (Value) >>> *Value = localValue; >>> Exit: >>> return error; >>> } >>> >>> >>> arrays of bytes are really the only way to control the layout. >>> Look at how GNU binutils works, for example... >>> Bitfields in C don't yield predictable/portable layout either. >>> >>> >>> - Jay >>> >>> > From: hosking at cs.purdue.edu >>> > Date: Thu, 23 Aug 2012 09:30:57 -0400 >>> > To: dragisha at m3w.org >>> > CC: m3devel at elegosoft.com >>> > Subject: Re: [M3devel] I know, I know... >>> > >>> > PACKED? >>> > >>> > Sent from my iPad >>> > >>> > On Aug 23, 2012, at 3:24 AM, Dragi?a Duri? wrote: >>> > >>> > > I know this will probably be very dense subject, but. What about: >>> > > >>> > > TSPacketHeader = <* ENDIAN = BIG*>RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > tErrInd, (* Transport Error Indicator *) >>> > > pusi: BITS 1 FOR BOOLEAN; (* Payload Unit Start Indicator *) >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pid: BITS 13 FOR PID; >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > cc: BITS 4 FOR Nibble; >>> > > END; >>> > > >>> > > Meaning: bit data is packed from left to right, and all multi-byte data is packed MSB first. >>> > > >>> > > Right now I am doing this, based on knowledge of my platform (little endian, x86_64), like this: >>> > > >>> > > TSPacketHeader = RECORD >>> > > sync: BITS 8 FOR [16_0..16_ff]; (* Always 0x47 *) >>> > > >>> > > pidHi: BITS 5 FOR [16_00..16_1f]; >>> > > transPrio: BITS 1 FOR [0..1]; >>> > > pusi, >>> > > tErrInd: BITS 1 FOR BOOLEAN; >>> > > >>> > > pidLo: BITS 8 FOR [16_00..16_ff]; >>> > > >>> > > cc: BITS 4 FOR Nibble; >>> > > afc: BITS 2 FOR [0..3]; (* 01 - no adaptation field, payload only *) >>> > > transScramControl: BITS 2 FOR [0..3]; (* 00 means no scrambling *) >>> > > END; >>> > > >>> > > And please don't tell me "write in C", because then I will just "offload" this problem to C preprocessor and still only hope for the best. Modula-3 is provably very adept to systems programming and I hope it can be more so. >>> > > >>> > > Thanks in advance.. >>> > > -- >>> > > Divided by a common language >>> > > >>> > > Dragi?a Duri? >>> > > dragisha at m3w.org >>> > > >>> > > >>> > > >>> > > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Aug 24 03:54:27 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 02:54:27 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120823230744.509A61A2094@async.async.caltech.edu> Message-ID: <1345773267.73869.YahooMailClassic@web29703.mail.ird.yahoo.com> Hi all: No, I don't agree there isn't compiler internal intended to break interface that's other story but we don't fail compiled IRs at least that we know anything (at least the DEC-SRC front end needs to recompile every interface separately of any Module, so you can change compilers and still work the same clock speed should be equal in whatever configuration). We should take advantage of that inherent parallelism? I guess but we would need a good distributed quake machine. Alignment and endianness are same story about I don't mind that I just care for our actual clock speed of our machines instead of thinking in polymorphic type instructions etc. We eat our own stuff, VAX style machine type, everything else is just? head ache: http://mail-index.netbsd.org/current-users/1999/04/27/0032.html Thanks in advance --- El jue, 23/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] I know, I know... Para: "Dragi?a Duri?" CC: m3devel at elegosoft.com Fecha: jueves, 23 de agosto, 2012 18:07 Well .M3WEB isn't defined in the Green Book.... Actually I take back what I said before.? If I was doing what you are describing from scratch, I'd write a Scheme program that takes an S-expression according to certain rules and generates both an interface defining a RECORD type Interface.T and packing/unpacking code in Modula-3 but following Jay's outline.? Then I'd write a template file that allows me to specify said Scheme file in m3makefile.? The advantage of this approach is that it doesn't rely on any compiler internals or special output files.? And it's at least as powerful as any pragmas you might want to add.? Plus easier to implement... Below is a very partial example of something similar... it defines a SQL database together with Modula-3 code for accessing the tables in that database (in very multithreaded and pipelined ways).? I only define two tables, you can imagine what the rest looks like.? It's for a real-time financial trading application, if you were wondering.? The Scheme is 253 lines and results in 8,828 lines of Modula-3 code and 361 lines of SQL.? The code generator is only 1,197 lines of Scheme and that is including liberal documentation.? (The system is similar to "Hibernate" but I suspect with more emphasis on certain kinds of performance that we cared very much about in the trading application, e.g., latency of piping data through the database.)? The whole thing is added to the program by adding a single line to m3makefile. ? ???Mika (define aux-mm-ordr-ctrl ? (make-table "aux_mm_ordr_ctrl" 'client ? ? (make-field "symbol"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-index (list "symbol")) ? ? (make-field "type"? ? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "expiry"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "exchange"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "currency"? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "action"? ? ? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "quantity"? ? ? ? 'double-precision 'not-null) ? ? (make-field "order_type"? ? ? 'varchar? ? ? ? ? 'not-null) ? ? (make-field "benchmark_price" 'double-precision) ? ? (make-field "time_allowance"? 'double-precision) ? ? (make-field "model_name"? ? ? 'varchar) ? ? (make-index (list "model_name")) ? ? (make-field "id"? ? ? ? ? ? ? 'integer) )) (define clean ? (make-table "clean" 'server ???(make-field "tabl"? ? ? ? ? 'varchar 'not-null) ???(make-field "rowid"? ? ? ???'integer 'not-null) ???(make-field "client"? ? ? ? 'varchar 'not-null) ? ? ???(make-index (list "client")) ???(make-index (list "tabl")) ???(make-index (list "tabl" "rowid")) ???(make-index (list "client" "tabl")))) ? ? (define gcoms ? (make-database "gcoms" ? ? ? ? ? ? ? ???clean ? ? ? ? ? ? ? ???code model instrument strategy model-data ? ? ? ? ? ? ? ???model-code ? ? ? ? ? ? ? ???ordr ordr_status ? ? ? ? ? ? ? ???output eror ? ? ? ? ? ? ? ???fill ? ? ? ? ? ? ? ???aux-instrument ? ? ? ? ? ? ? ???aux-ordr ? ? ? ? ? ? ? ???aux-ordr-params ? ? ? ? ? ? ? ???aux-market-segment ? ? ? ? ? ? ? ???aux-last-prices ? ? ? ? ? ? ? ???aux-mm-ordr-ctrl ? ? ? ? ? ? ? ???aux-mm-associations )) ? ? ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ??? (write-database-sql gcoms) (write-database-m3 gcoms) =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Type: multipart/alternative; >??? boundary="Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4" > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; >??? charset=utf-8 > >I have RTInfo module we developed years ago to parse .M3WEB. Good = >enough? :) But, not a solution, it is easier to write as I wrote in = >example I used to illustrate problem. > >GCC not obeying packing instructions is like=E2=80=A6 GCC not working in = >lot of cases. For some reason, I don't believe in it. > >Hint, gm2 is GCC based compiler. And supports very precise = >packing/aligning rules. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote: > >>=20 >> Well I don't think Jay said that bytes couldn't be unpacked in = >Modula-3 >> (you can just use Word for that). >>=20 >> But I think the point was that a compiler back-end might not = >necessarily >> obey your packing instructions? >>=20 >> Here's what I think you should do... it would be super useful, both to >> you and other people in the future.? And completely independent of >> compilers. >>=20 >> Write a something-or-other that uses m3tk to read your RECORD type >> spec, plus pragmas of your own choice (or comments, or auxiliary data >> from somewhere) and then generates code using Word.T that can process >> byte streams and return the RECORD you declared, in portable Modula-3. >> I doubt it's very difficult.? m3tk is cool! >>=20 >>? ? Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>>=20 >>> --Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E >>> Content-Type: multipart/alternative; >>> ??? boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122" >>>=20 >>>=20 >>> --Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122 >>> Content-Transfer-Encoding: quoted-printable >>> Content-Type: text/plain; >>> ??? charset=3Dwindows-1250 >>>=20 >>> I have a friend, working for large software company and he just = >recently =3D >>> ported some message router from SPARC to Linux/Intel=3D85 Lots of = >"network =3D >>> order" data, same as with my current projects. Network, communication = >in =3D >>> general, network order is everywhere. >>>=20 >>> Insisting on some god-given data ordering is a bit=3D85 What is nice = >word =3D >>> here? :) Outlandish? Outimeish? >>>=20 >>> This is not first time I started discussion like this here. Every = >single =3D >>> time Jay explains to me how Modula-3 cannot handle it. I am handling = >it, =3D >>> as I have shown in my example, with Modula-3 code. With a bit of = >effort =3D >>> I can make it almost-transparent (subfolder/platform) over various =3D >>> platforms. Of course, as I am developing a full product (not software = >to =3D >>> be run on arbitrary platform) I don't have to worry about too many =3D >>> platforms. One is enough here, but I still think Modula-3 can = >benefit, =3D >>> and a lot, if it supported explicit byte/bit = >ordering/packing/aligning =3D >>> pragmas. >>>=20 >>> Also, unlike GCC, pointer alignment in Modula-3 is 64bit on x86_64. = >It =3D >>> is 32bit in GCC, and x86_64 is totally happy with it. Wr have 64bit = >so =3D >>> we must write piece of software in C to be what? Compatible with = >API's =3D >>> all systems are compatible with without any hassle.=3D20 >>>=20 >>> TIA >>> -- >>> Divided by a common language >>>=20 >>> Dragi=3D9Aa Duri=3DE6 >>> dragisha at m3w.org >>>=20 > > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; >??? charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I = >have RTInfo module we developed years ago to parse .M3WEB. Good enough? = >:) But, not a solution, it is easier to write as I wrote in example I = >used to illustrate problem.

GCC not obeying = >packing instructions is like=E2=80=A6 GCC not working in lot of cases. = >For some reason, I don't believe in it.

Hint, = >gm2 is GCC based compiler. And supports very precise packing/aligning = >rules.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 23, 2012, at 9:51 PM, Mika Nystrom wrote:

class=3D"Apple-interchange-newline">
type=3D"cite">

Well I don't think Jay said that bytes couldn't = >be unpacked in Modula-3
(you can just use Word for that).

But = >I think the point was that a compiler back-end might not = >necessarily
obey your packing instructions?

Here's what I = >think you should do... it would be super useful, both to
you and = >other people in the future.  And completely independent = >of
compilers.

Write a something-or-other that uses m3tk to = >read your RECORD type
spec, plus pragmas of your own choice (or = >comments, or auxiliary data
from somewhere) and then generates code = >using Word.T that can process
byte streams and return the RECORD you = >declared, in portable Modula-3.
I doubt it's very difficult. = > m3tk is cool!

= >   Mika

=3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?= >=3D writes:

type=3D"cite">--Apple-Mail=3D_8871A9FB-3130-4B36-8524-F6549A5CEC9E
ockquote>
Content-Type: = >multipart/alternative;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >boundary=3D"Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122"
= >

type=3D"cite">
type=3D"cite">--Apple-Mail=3D_2314A877-555E-4BDE-8112-C875A2AE5122
ockquote>
Content-Transfer-Encoding: = >quoted-printable
Content-Type: = >text/plain;
class=3D"Apple-tab-span" style=3D"white-space:pre">??? = >charset=3Dwindows-1250
type=3D"cite">
I have a = >friend, working for large software company and he just recently = >=3D
ported some message router = >from SPARC to Linux/Intel=3D85 Lots of "network = >=3D
order" data, same as with = >my current projects. Network, communication in = >=3D
general, network order is = >everywhere.
type=3D"cite">
Insisting on = >some god-given data ordering is a bit=3D85 What is nice word = >=3D
here? :) Outlandish? = >Outimeish?
type=3D"cite">
This is not = >first time I started discussion like this here. Every single = >=3D
time Jay explains to me = >how Modula-3 cannot handle it. I am handling it, = >=3D
as I have shown in my = >example, with Modula-3 code. With a bit of effort = >=3D
I can make it = >almost-transparent (subfolder/platform) over various = >=3D
platforms. Of course, as I = >am developing a full product (not software to = >=3D
be run on arbitrary = >platform) I don't have to worry about too many = >=3D
platforms. One is enough = >here, but I still think Modula-3 can benefit, = >=3D
and a lot, if it supported = >explicit byte/bit ordering/packing/aligning = >=3D
type=3D"cite">pragmas.
type=3D"cite">
Also, unlike = >GCC, pointer alignment in Modula-3 is 64bit on x86_64. It = >=3D
is 32bit in GCC, and = >x86_64 is totally happy with it. Wr have 64bit so = >=3D
we must write piece of = >software in C to be what? Compatible with API's = >=3D
all systems are compatible = >with without any hassle.=3D20
type=3D"cite">
type=3D"cite">TIA
type=3D"cite">--
Divided by a = >common language
type=3D"cite">
Dragi=3D9Aa = >Duri=3DE6
href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org
ckquote = >type=3D"cite">

body>= > >--Apple-Mail=_97F57BF6-5A6D-479B-8575-88C17BAE93B4-- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; >??? filename=signature.asc >Content-Type: application/pgp-signature; >??? name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQNou3AAoJEJtljYXUJo8xhFQH/19LV/No97WCrAPygnWWZAa4 >DQDZtnFZuDhAxMut8QZWIZN/LikLsaC39Rd9YMr01oKRZ93gucjUigwad5cPfyUb >XUV0PNrjMr67TZo9zRcwuXZJp8FzvNDFsA1nOtorQbIPG3e5IaLNdLpihSGBJ3zt >vnd3wUjnPz56HbdDulA4kYkyxSkCQgJ457jI2DTR+wtsaqNREEl0Hzs1qlpDDXWC >ShRvTdiDGZV/TMHHwkkWRVjZkHHu3npPClhXzY0fBRXUu+k3NWzZbuzIhXofCJ9y >C+yLl7qKdtU5cF+JpObPReA2ZfarW7AJ7rLLHZJa6NCs75qtCzo68JWhY8MtF3Y= >=OiB3 >-----END PGP SIGNATURE----- > >--Apple-Mail=_6458BB99-E0EF-4A10-A66A-491737830F8E-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Aug 24 16:00:52 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 24 Aug 2012 10:00:52 -0400 Subject: [M3devel] I know, I know... In-Reply-To: <20120823195115.80C121A2094@async.async.caltech.edu> References: <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu> <25147774-5A4A-4793-921A-2C9101273773@m3w.org> <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org> <20120823195115.80C121A2094@async.async.caltech.edu> Message-ID: <20120824140052.GA8309@topoi.pooq.com> On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future. And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice. If they are ignored on an implementation that doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone. Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away. It may be a deficiency we can live with. -- hendrik From dabenavidesd at yahoo.es Fri Aug 24 17:07:35 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Aug 2012 16:07:35 +0100 (BST) Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> Message-ID: <1345820855.90388.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: In SPIN M3 you could VIEW any type as a bit string, but they also had a different compiler, so it's difficult to compare, in any case the thing written like that is UNSAFE in the data abstraction sense and so it's kind of easy see? that structure pattern in any other language with UNSAFE patterns. I like more structural typing in the sense if all are different you can't assign them, if you can then it's because they are not mismatched and obey the semantic rules of the language. Interesting combination of both would make a better RT though. Thanks in advance --- El vie, 24/8/12, Hendrik Boom escribi?: De: Hendrik Boom Asunto: Re: [M3devel] I know, I know... Para: m3devel at elegosoft.com Fecha: viernes, 24 de agosto, 2012 09:00 On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > (you can just use Word for that). > > But I think the point was that a compiler back-end might not necessarily > obey your packing instructions? > > Here's what I think you should do... it would be super useful, both to > you and other people in the future.? And completely independent of > compilers. > > Write a something-or-other that uses m3tk to read your RECORD type > spec, plus pragmas Pragmas are definitely the wrong choice.? If they are ignored on an implementation that? doesn't recognise them, this will be the wrong semantics. Coding all the extraction and insertion functions is obscure, picky, and error-prone.? Bit-mapping is one of the reasons people use systems languages, and the lack of it is a deficiency in Modula 3. Not that I'm advocating rewriting the compiler right away.? It may be a deficiency we can live with. -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:43:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:43:44 +0000 Subject: [M3devel] I know, I know... In-Reply-To: <20120824140052.GA8309@topoi.pooq.com> References: , <97686E7C-E80D-4357-BF60-49C0FA2C816E@cs.purdue.edu>, , <25147774-5A4A-4793-921A-2C9101273773@m3w.org>, <24E137D4-F18E-47B9-8BFB-462337110DC3@m3w.org>, <20120823195115.80C121A2094@async.async.caltech.edu>, <20120824140052.GA8309@topoi.pooq.com> Message-ID: Modula-3 has bit-mapping. Encoding it in functions that takes offsets and sizes is not hard. Using the language features gives you a choice of either - portable layout that doesn't likely match C - unportable layout that might match C At least unportable in terms of endianness -- again, for better and worse, see how we pick apart floats. It works, but the declarations are duplicated for little endian vs. big endian. Such knowledge does traditionally live in a compiler and its "core" libraries/runtime, but I'd still like to minimize it, maybe eliminate it. Neither is a ggeat choice I think. The compiler had tried to match target-specific layout/alignment rules. Whether it got them correct, I don't know. I relaxed the code because it was hard to know if it was right or wrong and just didn't seem worth maintaining -- i.e. porting to new systems. - Jay > Date: Fri, 24 Aug 2012 10:00:52 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] I know, I know... > > On Thu, Aug 23, 2012 at 12:51:15PM -0700, Mika Nystrom wrote: > > > > Well I don't think Jay said that bytes couldn't be unpacked in Modula-3 > > (you can just use Word for that). > > > > But I think the point was that a compiler back-end might not necessarily > > obey your packing instructions? > > > > Here's what I think you should do... it would be super useful, both to > > you and other people in the future. And completely independent of > > compilers. > > > > Write a something-or-other that uses m3tk to read your RECORD type > > spec, plus pragmas > > Pragmas are definitely the wrong choice. If they are ignored on an > implementation that doesn't recognise them, this will be the wrong > semantics. > > Coding all the extraction and insertion functions is obscure, picky, and > error-prone. Bit-mapping is one of the reasons people use systems > languages, and the lack of it is a deficiency in Modula 3. > > Not that I'm advocating rewriting the compiler right away. It may be a > deficiency we can live with. > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 24 19:48:02 2012 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Aug 2012 17:48:02 +0000 Subject: [M3devel] Modula-3 bitfields Message-ID: Given: struct foo { union { struct { unsigned a : 16; unsigned b : 16; } s; unsigned c; } u; } foo = { {1, 2} }; printf("%X %X\n", c.u.s.a, c.u.s.b); What do people expect? Do people really have a mental model as to what the layout rules are for bitfields? Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? Furthermore, if presented with: struct foo { unsigned a : 16; unsigned b : 16; }; void F1(struct foo* f); printf("%X %X\n", f->a, f->b); Foo = RECORD a, b: BITS 16 FOR INTEGER END; <*EXTERNAL F1*> PROCEDURE F1(VAR foo: Foo); VAR f := Foo{1,2}; F1(f); What do people expect? That is, 1) Does anyone really internalize the layout rules for bitfields? 2) Does anyone really need them to line up with C? I ask because, it seems to me that in m3front/m3middle, if Little_endian were removed and assumed either always true or always false, everything would actually just work. The only thing that would go wrong is 1) interoperability with C..when there are bitfields 2) including in m3core where floats are picked apart 2b) But that's kind of good -- we could remove the little vs. big endian distinction there. See, I suspect but I am not certain the layout rules are actually well known to some small set of people, and they got encoded in m3front, in order for easier interoperability with C. Repeat the above exercises for more complicated examples, i.e. bitcounts that aren't multiples of 16. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri Aug 24 19:58:08 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 19:58:08 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: Message-ID: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> What mental model? We have network order. Everything network is bit streams, from left to right, most significant bits/bytes. No need for mental model here, any communication oriented ISO, ETSI, ITU standard? I don't understand why you insist on C syntax here. C has its own problems and reasons why it's at war with most things mental. Please refer to my initial example instead. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 7:48 PM, Jay K wrote: > printf("%X %X\n", c.u.s.a, c.u.s.b); > What do people expect? > Do people really have a mental model as to what the layout rules are for bitfields? > Are they obvious enough that every C compiler writer has actually implemented them the same and they become in a sense portable? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 20:10:35 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 11:10:35 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> Message-ID: <20120824181035.4EEFE1A207D@async.async.caltech.edu> The Green Book doesn't actually define alignment rules does it? Your example as I recall also specified precisely 32 bits. I can think of lots of ways things can go wrong. Why would you want to tie the hands of the compiler writer to a particular layout? Or clutter the interface with lots of pragmas... Yeah OK there was a day when that was the way to do things.... I think that time is past. The method I suggested could take advantage of situations where you know precisely how to construct the packed types. If you wanted. But it would also be entirely portable and independent of compilers. Heck you could generate C code at the same time and have free cross-language interoperability. (Or any other sets of languages.) You could also split bitfields in funny ways... Why would you do things any other way, really? Here's an example. A PCI Express DLLP is 32 bits of payload followed by a 16-bit CRC. Should I be able to write this as follows: DLLP = RECORD payload : BITS 32 FOR [0..16_ffffffff]; crc : BITS 16 FOR [0..16_ffff]; END; What then is the layout of ARRAY [0..1] OF DLLP; ? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Type: multipart/alternative; > boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >What mental model? > >We have network order. Everything network is bit streams, from left to = >right, most significant bits/bytes. No need for mental model here, any = >communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 > >I don't understand why you insist on C syntax here. C has its own = >problems and reasons why it's at war with most things mental. Please = >refer to my initial example instead. > >-- >Divided by a common language > >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > > >On Aug 24, 2012, at 7:48 PM, Jay K wrote: > >> printf("%X %X\n", c.u.s.a, c.u.s.b); >> What do people expect? >> Do people really have a mental model as to what the layout rules are = >for bitfields? >> Are they obvious enough that every C compiler writer has actually = >implemented them the same and they become in a sense portable?=20 > > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >mental model?

We have network order. Everything = >network is bit streams, from left to right, most significant bits/bytes. = >No need for mental model here, any communication oriented ISO, ETSI, ITU = >standard=E2=80=A6 

I don't understand why = >you insist on C syntax here. C has its own problems and reasons why it's = >at war with most things mental. Please refer to my initial example = >instead.

>color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">
--
Divided by a = >common language

Dragi=C5=A1a = >Duri=C4=87

= >


class=3D"Apple-interchange-newline"> >
>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

class=3D"Apple-interchange-newline">
class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >expect?
Do people really have a mental model as to what the layout = >rules are for bitfields?
Are they obvious enough that every C = >compiler writer has actually implemented them the same and they become = >in a sense portable?class=3D"Apple-converted-space"> 
div>
= > >--Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > >iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >=1GwZ >-----END PGP SIGNATURE----- > >--Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- From dragisha at m3w.org Fri Aug 24 21:20:03 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:20:03 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> As people in the world are divided in three groups, ones who can count and everybody else? So are we. I am in this group with this fancy idea of Modula-3 being system programming language. You don't agree? On Aug 24, 2012, at 8:10 PM, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Most probably. That's why I mentioned prgma in the first place. > > Your example as I recall also specified precisely 32 bits. > > I can think of lots of ways things can go wrong. > > Why would you want to tie the hands of the compiler writer to a particular > layout? Or clutter the interface with lots of pragmas? Pragma is good way to make things optional for some platforms, and to not change language definition. Platform specific, and working on AMD64 for example, is good enough. > > Yeah OK there was a day when that was the way to do things.... I think > that time is past. The method I suggested could take advantage of > situations where you know precisely how to construct the packed types. > If you wanted. But it would also be entirely portable and independent > of compilers. Heck you could generate C code at the same time and have > free cross-language interoperability. (Or any other sets of languages.) > > You could also split bitfields in funny ways... > > Why would you do things any other way, really? Being a Modula-2|3 programmer for 24 years obviously did funny things to me :). Like, I really started to like how high-level programs look like, esp ones doing various low-level stuff :). I like to think abstract, to write high-level, readable programs. Ones I don't have problems maintaining for prolonged time intervals. I don't need anything funny. I just need a way to represent, in a readable manner, standard communication data structures. So I can continue with my trade without C, > > Here's an example. A PCI Express DLLP is 32 bits of payload followed by > a 16-bit CRC. Should I be able to write this as follows: > > DLLP = RECORD > payload : BITS 32 FOR [0..16_ffffffff]; > crc : BITS 16 FOR [0..16_ffff]; > END; > > What then is the layout of > > ARRAY [0..1] OF DLLP; > Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? > ? > > Mika dd > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E" >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> What mental model? >> >> We have network order. Everything network is bit streams, from left to = >> right, most significant bits/bytes. No need for mental model here, any = >> communication oriented ISO, ETSI, ITU standard=E2=80=A6=20 >> >> I don't understand why you insist on C syntax here. C has its own = >> problems and reasons why it's at war with most things mental. Please = >> refer to my initial example instead. >> >> -- >> Divided by a common language >> >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> >> On Aug 24, 2012, at 7:48 PM, Jay K wrote: >> >>> printf("%X %X\n", c.u.s.a, c.u.s.b); >>> What do people expect? >>> Do people really have a mental model as to what the layout rules are = >> for bitfields? >>> Are they obvious enough that every C compiler writer has actually = >> implemented them the same and they become in a sense portable?=20 >> >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">What = >> mental model?

We have network order. Everything = >> network is bit streams, from left to right, most significant bits/bytes. = >> No need for mental model here, any communication oriented ISO, ETSI, ITU = >> standard=E2=80=A6 

I don't understand why = >> you insist on C syntax here. C has its own problems and reasons why it's = >> at war with most things mental. Please refer to my initial example = >> instead.

>> > color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">
--
Divided by a = >> common language

Dragi=C5=A1a = >> Duri=C4=87

= >>


> class=3D"Apple-interchange-newline"> >>
>>
On Aug 24, 2012, at 7:48 PM, Jay K wrote:

> class=3D"Apple-interchange-newline">
> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"font-family: Calibri; font-size: = >> 16px; ">printf("%X %X\n", c.u.s.a, c.u.s.b);
What do people = >> expect?
Do people really have a mental model as to what the layout = >> rules are for bitfields?
Are they obvious enough that every C = >> compiler writer has actually implemented them the same and they become = >> in a sense portable?> class=3D"Apple-converted-space"> 
> div>
= >> >> --Apple-Mail=_01BB8F3E-2A9D-4270-9D92-38991735317E-- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A >> Content-Transfer-Encoding: 7bit >> Content-Disposition: attachment; >> filename=signature.asc >> Content-Type: application/pgp-signature; >> name=signature.asc >> Content-Description: Message signed with OpenPGP using GPGMail >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.18 (Darwin) >> >> iQEcBAEBAgAGBQJQN8DnAAoJEJtljYXUJo8xZsIH/3eJ6oSYCbMxR0osICTBayug >> fWeDasX0FPM1/BglqM5pS5DzITUrVvSp1ldXUr6Na1RaPKc9kUvs521YNHG6yajs >> tRO0A6T1subyKNjscPrYYUueT2EHABrg0qfVD6NeUHxeE35304CztXoNx0HMhKtx >> 5CfNORPHoGl1utyKxj0OabDXnHDb3b34sgmHWBSBlNc2Mk1lBBcss3UJT5p2o8yq >> M3p19YhPst/YQove/21UYEuDDJB4TiLl2rQaV4h5gPPTQLXGzF35VWRNeKsrWf8f >> VhhwckLhgU6cvmLUaXacIulVdpdOfaavG2cg+62ptWNtIHv7frlSHsDxTjHYavk= >> =1GwZ >> -----END PGP SIGNATURE----- >> >> --Apple-Mail=_91577572-D1CD-4E97-A777-A61FB5FE2B5A-- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Fri Aug 24 21:28:59 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 12:28:59 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> Message-ID: <20120824192859.C3B801A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: ... >to me :). Like, I really started to like how high-level programs look = >like, esp ones doing various low-level stuff :). I like to think = >abstract, to write high-level, readable programs. Ones I don't have = >problems maintaining for prolonged time intervals. > > I don't need anything funny. I just need a way to represent, in a = >readable manner, standard communication data structures. So I can = >continue with my trade without C, > >>=20 >> Here's an example. A PCI Express DLLP is 32 bits of payload followed = >by >> a 16-bit CRC. Should I be able to write this as follows: >>=20 >> DLLP =3D RECORD >> payload : BITS 32 FOR [0..16_ffffffff]; >> crc : BITS 16 FOR [0..16_ffff]; >> END; >>=20 >> What then is the layout of=20 >>=20 >> ARRAY [0..1] OF DLLP; >>=20 > >Maybe you meant to write: ARRAY [0..1] OF BITS 48 FOR DLLP; ? And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika From dragisha at m3w.org Fri Aug 24 21:53:44 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 24 Aug 2012 21:53:44 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824192859.C3B801A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> Message-ID: <14795BC4-4830-442C-A674-633197203234@m3w.org> What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: > > And what does that mean, exactly? No gaps allowed? What does it > mean, in particular, on a little-endian machine? > > What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > > What I am proposing isn't particularly difficult, it's not unportable, > either. And much more powerful than any sets of pragmas. Not tied to > your one architecture. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 05:06:06 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 03:06:06 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <14795BC4-4830-442C-A674-633197203234@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: > What mental model? > We have network order. Everything network is bit streams, from left to right, most > significant bits/bytes. Does any Modula-2 or Modula-3 or C compiler follow this model?For a little endian target? I'm skeptical.If it really is obvious, maybe we can make cm3 follow it?Even if it breaks bitfield interop with C?I would like to have m3front and M3C.m3 not have any target-endian dependentness.I do believe it would break pickles, and some floating point code in m3core/libm3. ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). In particular, standard C only allows bit fields to be of type "int" or "unsigned".However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? What does this look like: struct { unsigned a : 1; unsigned b : 32;};? - Jay From: dragisha at m3w.org Date: Fri, 24 Aug 2012 21:53:44 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields What you are proposing is to do a compiler-level job with a library. Just because such a library exists does not mean it invalidates every other approach. And we are talking about one of most complicated pieces of Modula-3 system, comparable in complexity with compiler. Of course, there are other approaches too, like some kind of interface language, to be processed into Modula-3 source code by cm3 build system. Your latest example is good illustration for my proposal too. Another simple example to show a need for exact bit packing layout method. Thank you! --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 24, 2012, at 9:28 PM, Mika Nystrom wrote: And what does that mean, exactly? No gaps allowed? What does it mean, in particular, on a little-endian machine? What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? What I am proposing isn't particularly difficult, it's not unportable, either. And much more powerful than any sets of pragmas. Not tied to your one architecture. Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Aug 25 07:35:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 24 Aug 2012 22:35:51 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: <20120825053551.CB4BD1A207D@async.async.caltech.edu> Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. Mika Jay K writes: >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/plain; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > What mental model? > We have network order. Everything network is bit st= >reams=2C from left to right=2C most > significant bits/bytes. Does any Mod= >ula-2 or Modula-3 or C compiler follow this model?For a little endian targe= >t? I'm skeptical.If it really is obvious=2C maybe we can make cm3 follow it= >?Even if it breaks bitfield interop with C?I would like to have m3front and= > M3C.m3 not have any target-endian dependentness.I do believe it would brea= >k pickles=2C and some floating point code in m3core/libm3. ..because..you = >see.. I think there some mental model in C compiler writer's heads...that c= >m3 tries to adhere to=2C but which I might have diverged a bit from (look a= >t the history of Target.m3). In particular=2C standard C only allows bit f= >ields to be of type "int" or "unsigned".However it is a common extension to= > allow the type of a bitfield to also unsigned/signed char=2C short=2C long= >=2C and long long=2C and this does I believe affect the layout. Does that f= >it your mental model? In your model=2C must a 32bit bit field be aligned on= > a 32bit boundary? What does this look like: struct { unsigned a : 1=3B u= >nsigned b : 32=3B}=3B? - Jay > From: dragisha at m3w.org >Date: Fri=2C 24 Aug 2012 21:53:44 +0200 >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: Re: [M3devel] Modula-3 bitfields > >What you are proposing is to do a compiler-level job with a library. Just b= >ecause such a library exists does not mean it invalidates every other appro= >ach. And we are talking about one of most complicated pieces of Modula-3 sy= >stem=2C comparable in complexity with compiler. Of course=2C there are othe= >r approaches too=2C like some kind of interface language=2C to be processed= > into Modula-3 source code by cm3 build system. >Your latest example is good illustration for my proposal too. Another simpl= >e example to show a need for exact bit packing layout method. Thank you! >=0A= >--Divided by a common language >Dragi=B9a Duri=E6dragisha at m3w.org >=0A= >=0A= > >On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote: >And what does that mean=2C exactly? No gaps allowed? What does it >mean=2C in particular=2C on a little-endian machine? > >What about ARRAY [0..1] OF BITS 31 FOR [0..16_8fffffff] ? > >What I am proposing isn't particularly difficult=2C it's not unportable=2C >either. And much more powerful than any sets of pragmas. Not tied to >your one architecture. > > Mika > = > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_ >Content-Type: text/html; charset="iso-8859-2" >Content-Transfer-Encoding: quoted-printable > > > > >
 =3B>=3B What mental model= >?
 =3B>=3B We have network order. Everything network is bit s= >treams=2C from left to right=2C most
 =3B>=3B significant b= >its/bytes.
 =3B
 =3B
Does any Modula-= >2 or Modula-3 or C compiler follow this model?
For a little endia= >n target? I'm skeptical.
If it really is obvious=2C maybe we can = >make cm3 follow it?
Even if it breaks bitfield interop with C?iv>
I would like to have m3front and M3C.m3 not have any target-endian = >dependentness.
I do believe it would break pickles=2C and some fl= >oating point code in m3core/libm3.
 =3B ..because..you see.. = >I think there some mental model in C compiler writer's heads...that cm3 tri= >es to adhere to=2C but which I might have diverged a bit from (look at the = >history of Target.m3).
 =3B
 =3B
In p= >articular=2C standard C only allows bit fields to be of type "int" or "unsi= >gned".
However it is a common extension to allow the type of a bi= >tfield to also unsigned/signed char=2C short=2C long=2C and long long=2C an= >d this does I believe affect the layout. Does that fit your mental model? I= >n your model=2C must a 32bit bit field be aligned on a 32bit boundary?>
 =3B
What does this look like:
 =3B
= >
struct {
 =3B unsigned a : 1=3B
 =3B unsig= >ned b : 32=3B
}=3B
?
 =3B
 = >=3B
 =3B- Jay
 =3B
older">

From: dragisha at m3w.org
Date: Fri=2C= > 24 Aug 2012 21:53:44 +0200
To: mika at async.caltech.edu
CC: m3devel at el= >egosoft.com
Subject: Re: [M3devel] Modula-3 bitfields

What you ar= >e proposing is to do a compiler-level job with a library. Just because such= > a library exists does not mean it invalidates every other approach. And we= > are talking about one of most complicated pieces of Modula-3 system=2C com= >parable in complexity with compiler. Of course=2C there are other approache= >s too=2C like some kind of interface language=2C to be processed into Modul= >a-3 source code by cm3 build system.

Your latest ex= >ample is good illustration for my proposal too. Another simple example to s= >how a need for exact bit packing layout method. Thank you!

div>
=0A= >nt: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B white-space: norm= >al=3B border-collapse: separate=3B orphans: 2=3B widows: 2=3B" class=3D"ecx= >Apple-style-span"> none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacing: 0px=3B= > white-space: normal=3B border-collapse: separate=3B orphans: 2=3B widows: = >2=3B" class=3D"ecxApple-style-span">
>
--
Divided by a common language

Dr= >agi=B9a Duri=E6


newline">

=0A= >
=0A= >
On Aug 24=2C 2012=2C at 9:28 PM=2C Mika Nystrom wrote:
<= >br class=3D"ecxApple-interchange-newline">
ransform: none=3B text-indent: 0px=3B letter-spacing: normal=3B word-spacin= >g: 0px=3B white-space: normal=3B border-collapse: separate=3B orphans: 2=3B= > widows: 2=3B" class=3D"ecxApple-style-span">
And what does that mean=2C= > exactly?  =3BNo gaps allowed?  =3BWhat does it
mean=2C in parti= >cular=2C on a little-endian machine?

What about ARRAY [0..1] OF BITS= > 31 FOR [0..16_8fffffff] ?

What I am proposing isn't particularly di= >fficult=2C it's not unportable=2C
either.  =3BAnd much more powerful= > than any sets of pragmas.  =3BNot tied to
your one architecture.>
 =3B =3B =3B =3BMika

v>
>= > >--_bf70e44f-528a-4dc1-b84a-af7891d54830_-- From dragisha at m3w.org Sat Aug 25 08:29:12 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:29:12 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> Message-ID: Did I said in my first post, pragma? With pragma you don't activate a behavior unless you need something pragma provides. Locally, in pragma's scope. So you will not break anything, anywhere outside pragma's scope. gm2 is very precise with bitpacking and aligning. I did not check for byte/bit order, or anything gm2 recently, too much work (most of it in Modula-3). As this is perfectly logical system programming requirement I do not doubt Gaius will be positive to such proposal. What good is bitfield interop when our pointers are 64bit aligned and GCC's are 32bit? And when every interop question is answered with "write critical piece in C" ? Proposed pragma would not break anything, it is local behavior. I believe in "mental model", but I also believe in evolution. Do you? :) All of us have decades of experience accumulated, and my decades changed my models, as slow as it is. I see this pattern, I see this need. I see solution through alternate bit/byte packing into structures inside pragma scope. As for your example, outside of proposed pragma scope it will be 32 bit aligned. But, if it is a 64 bit value - in GCC it will be 32bit aligned, while cm3 will (most probably) align it to 64bit. There goes interop. Ooops. TIA -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 5:06 AM, Jay K wrote: > > What mental model? > > We have network order. Everything network is bit streams, from left to right, most > > significant bits/bytes. > > > Does any Modula-2 or Modula-3 or C compiler follow this model? > For a little endian target? I'm skeptical. > If it really is obvious, maybe we can make cm3 follow it? > Even if it breaks bitfield interop with C? > I would like to have m3front and M3C.m3 not have any target-endian dependentness. > I do believe it would break pickles, and some floating point code in m3core/libm3. > ..because..you see.. I think there some mental model in C compiler writer's heads...that cm3 tries to adhere to, but which I might have diverged a bit from (look at the history of Target.m3). > > > In particular, standard C only allows bit fields to be of type "int" or "unsigned". > However it is a common extension to allow the type of a bitfield to also unsigned/signed char, short, long, and long long, and this does I believe affect the layout. Does that fit your mental model? In your model, must a 32bit bit field be aligned on a 32bit boundary? > > What does this look like: > > struct { > unsigned a : 1; > unsigned b : 32; > }; > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sat Aug 25 08:31:16 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 08:31:16 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825053551.CB4BD1A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> Message-ID: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 09:22:44 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 07:22:44 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: 1) Most likely, whatever you want, you can get with Modula-3. You can at least attain the compactness you want. How you match up with some specified protocol, independently implementable e.g. with C and shifting/masking, less certain. 2) If this is for some network/persistant I/O, the result will not be portable. At least due to endianness. Possibly due to alignment. 3) I claim that interop with C remains a strong feature of Modula-3, but not necessarily regarding bitfields. There is some attempt at such interop, but I'd just as soon ignore it. I'd even give an error if types with bitfields are used with <* external *> functions. For my libc interop, I'm very conservative -- records contain just longint and integer, sorted by size -- there is never any padding, except maybe at the end. I looked a bit through "windows.i3" and "xlib.i3" and decided they seem pretty save too. 3) I am faced with generating target-independent C. This is the worry/doubt I inject/tangent into your line of questioning. You ask about one thing, I bring up something else (kind of like Daniel..) m3front checks endian, making it almost impossible. I fear there isn't a good answer here. I kind of would just like to a) remove the check b) adjust {libm3, m3core}/float accordingly. Alternatively, change m3front/m3cg some. Have the result say: #ifdef LITTLE_ENDIAN something #else something else #endif Perhaps the endian check can be removed if the target platform is "C"? Relevant aside: "By default", in C#, layout is undefined. It could sort the fields by size/name, I think, and be valid. Or put stuff near each other based on runtime profiling. Is Modula-3 this abstracted too?? But there are at least two ways to control the layout. You can assert that it must be "sequential". You can give actual offsets. I don't think C# even has bitfields though. - Jay Subject: Re: [M3devel] Modula-3 bitfields From: dragisha at m3w.org Date: Sat, 25 Aug 2012 08:31:16 +0200 CC: jay.krell at cornell.edu; m3devel at elegosoft.com To: mika at async.caltech.edu I see no problem with this. As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote:Yeah I think Modula-3 historically tries to stay as close as possible to an implied mapping to C. It's definitely *not* big-endian on a little-endian machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 13:22:17 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 13:22:17 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org>, <20120824192859.C3B801A207D@async.async.caltech.edu>, <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu>, <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <21696995-DEE3-4DD7-AAB5-13C0EC62316B@m3w.org> On Aug 25, 2012, at 9:22 AM, Jay K wrote: > 1) Most likely, whatever you want, you can get with Modula-3. > You can at least attain the compactness you want. > How you match up with some specified protocol, independently > implementable e.g. with C and shifting/masking, less certain. Modula-3 can do everything needed, in ad-hoc and generally totally unportable manner :). > > > 2) If this is for some network/persistant I/O, the result will > not be portable. At least due to endianness. Possibly due to alignment. Not if we have means to force both endianness of data and alignment. > > > 3) I claim that interop with C remains a strong feature of Modula-3, > but not necessarily regarding bitfields. There is some attempt at such > interop, but I'd just as soon ignore it. I'd even give an error > if types with bitfields are used with <* external *> functions. I don't need much bitfield related interop with C. 32bit aligned pointers remain on my wish list :). > > > For my libc interop, I'm very conservative -- records contain just > longint and integer, sorted by size -- there is never any padding, > except maybe at the end. > I looked a bit through "windows.i3" and "xlib.i3" and decided > they seem pretty save too. libc interop is great, and I depend a lot on it. Thank you. > 3) I am faced with generating target-independent C. > This is the worry/doubt I inject/tangent into your line of questioning. > You ask about one thing, I bring up something else (kind of like Daniel..) :) This realy is a tangent. As my proposal was (localized to single structures) pragma, I don't see a link here. dd > m3front checks endian, making it almost impossible. > I fear there isn't a good answer here. > I kind of would just like to a) remove the check b) adjust > {libm3, m3core}/float accordingly. > Alternatively, change m3front/m3cg some. > Have the result say: > > > #ifdef LITTLE_ENDIAN > something > #else > something else > #endif > > > Perhaps the endian check can be removed if the target platform is "C"? > > > Relevant aside: > "By default", in C#, layout is undefined. > It could sort the fields by size/name, I think, and be valid. > Or put stuff near each other based on runtime profiling. > Is Modula-3 this abstracted too?? > But there are at least two ways to control the layout. > You can assert that it must be "sequential". > You can give actual offsets. > I don't think C# even has bitfields though. > > > - Jay > > > > > Subject: Re: [M3devel] Modula-3 bitfields > From: dragisha at m3w.org > Date: Sat, 25 Aug 2012 08:31:16 +0200 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: mika at async.caltech.edu > > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to C". Also - proposed pragma is local behavior. Not active unless you need it, but when you need it you will be really grateful for it :). > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 7:35 AM, Mika Nystrom wrote: > > Yeah I think Modula-3 historically tries to stay as close as possible to > an implied mapping to C. It's definitely *not* big-endian on a little-endian > machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hendrik at topoi.pooq.com Sat Aug 25 21:05:48 2012 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 25 Aug 2012 15:05:48 -0400 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <4ACC8FB5-B392-48D3-AABF-74E243E871AD@m3w.org> <20120824192859.C3B801A207D@async.async.caltech.edu> <14795BC4-4830-442C-A674-633197203234@m3w.org> <20120825053551.CB4BD1A207D@async.async.caltech.edu> <473900A9-10DE-4E45-9D62-7A33D027AF33@m3w.org> Message-ID: <20120825190548.GA9036@topoi.pooq.com> On Sat, Aug 25, 2012 at 08:31:16AM +0200, Dragi?a Duri? wrote: > I see no problem with this. > > As I have illustrated, we are already apart from "implied mapping to > C". Also - proposed pragma is local behavior. Not active unless you > need it, but when you need it you will be really grateful for it :). To adequately describe data structures defined elsewhere (as in communications standards) we need to be able to specify alignment, width, padding, and endianness. It doesn't need to be global for all the data in a program -- in fact, it had better not. It should be specified explicitly for each data structure or data item for which it matters. And it's a crucial part of the semantics of the program, not just something that the compiler should handle if it happens to recognise a pragma -- it needs to be part of the panguage proper. How the compiler chooses to store other data, which is not so draconically constrained, is entirely up to it. That said, it's still an open question whether to do something like this at all. But if it is to be done, it has to be done right. -- hendrik From microcode at zoho.com Sat Aug 25 21:13:17 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sat, 25 Aug 2012 19:13:17 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120824181035.4EEFE1A207D@async.async.caltech.edu> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> Message-ID: <20120825191317.GA3747@zoho.com> On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: > The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) From dragisha at m3w.org Sat Aug 25 22:08:46 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:08:46 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825191317.GA3747@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> Message-ID: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Systems Programming with Modula 3 Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote: > On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: >> The Green Book doesn't actually define alignment rules does it? > > Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison > but it's white and grimy (looks like it was a garage copy!) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Sat Aug 25 22:20:56 2012 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Aug 2012 20:20:56 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: Microsoft C and gcc have a variety of ways to control this stuff. command line for the entire compilation #pragma pack/push/pop for a run of code __declspec(align) on individual types __unaligned on individual fields various __attribute__ probably on types and fields Which brings me back to ...array of bytes, very clear, very portable... If you need compactness but not exact layout, C and Modula-3 do fine. If you need a portable predictable layout, other than an array of bytes, they both largely fail. - Jay From: dragisha at m3w.org Date: Sat, 25 Aug 2012 22:08:46 +0200 To: microcode at zoho.com CC: m3devel at elegosoft.com Subject: Re: [M3devel] Modula-3 bitfields Systems Programming with Modula 3Edited by Greg Nelson Modula-3 bible. Most of material is available online in various SRC reports (now on HP's ftp servers) --Divided by a common language Dragi?a Duri?dragisha at m3w.org On Aug 25, 2012, at 9:13 PM, microcode at zoho.com wrote:On Fri, Aug 24, 2012 at 11:10:35AM -0700, Mika Nystrom wrote: The Green Book doesn't actually define alignment rules does it? Sorry, what's the Green Book? I just received a copy of Modula-3 by Harbison but it's white and grimy (looks like it was a garage copy!) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Aug 25 22:56:41 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 25 Aug 2012 22:56:41 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> I am in no doubt about general inadequacies of C et al? What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 10:20 PM, Jay K wrote: > Which brings me back to ...array of bytes, very clear, very portable... > > If you need compactness but not exact layout, C and Modula-3 do fine. > If you need a portable predictable layout, other than an array of bytes, they both largely fail. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Aug 25 23:55:19 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 25 Aug 2012 14:55:19 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: <20120825215519.BA0431A207D@async.async.caltech.edu> =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" > > >--Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1250 > >I am in no doubt about general inadequacies of C et al=85 > >What I hate is an idea of setters and getters for my bitfields. With = >pragma, compiler generated load/store will behave as needed. Everything = >transparent and just-working. Until you have some data that is PDP-endian :-) Or want to split a field across a boundary that you're not supposed to be able to split a field across. Or have a field that's semantically scattered/gathered. Can your proposed scheme represent IEEE floating point? I agree it is a very useful idea, I agree it has to be done "right". But my definition of doing it "right" would be so demanding you'd be crazy to do it in the compiler.... Mika > >-- >Divided by a common language > >Dragi=9Aa Duri=E6 >dragisha at m3w.org > From jay.krell at cornell.edu Sun Aug 26 04:59:26 2012 From: jay.krell at cornell.edu (Jay) Date: Sat, 25 Aug 2012 19:59:26 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> Message-ID: Other languages let you have "properties" that let you hide get/set functions behind field access syntax... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 25, 2012, at 1:56 PM, Dragi?a Duri? wrote: > I am in no doubt about general inadequacies of C et al? > > What I hate is an idea of setters and getters for my bitfields. With pragma, compiler generated load/store will behave as needed. Everything transparent and just-working. > > -- > Divided by a common language > > Dragi?a Duri? > dragisha at m3w.org > > > > > On Aug 25, 2012, at 10:20 PM, Jay K wrote: > >> Which brings me back to ...array of bytes, very clear, very portable... >> >> If you need compactness but not exact layout, C and Modula-3 do fine. >> If you need a portable predictable layout, other than an array of bytes, they both largely fail. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 26 06:56:59 2012 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 26 Aug 2012 04:56:59 +0000 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> Message-ID: <20120826045659.GA27610@zoho.com> On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- _ _ ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. From dragisha at m3w.org Sun Aug 26 09:03:14 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:03:14 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120825215519.BA0431A207D@async.async.caltech.edu> References: , <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org>, <20120824181035.4EEFE1A207D@async.async.caltech.edu>, <20120825191317.GA3747@zoho.com>, <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <05C8CBEA-A84B-4A19-9A2F-49800BED6EA7@m3w.org> <20120825215519.BA0431A207D@async.async.caltech.edu> Message-ID: <6AB35AF7-0D1A-40EA-B03A-E270B393CC8A@m3w.org> Right and simple are two faces of that coin. Every long walk has its first step. And so on :). -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 25, 2012, at 11:55 PM, Mika Nystrom wrote: > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_556D4F87-7FB1-46B0-B230-8D253F9EF1B5 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC" >> >> >> --Apple-Mail=_1902AEBB-876B-49B9-8A8F-13F98C1F82AC >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1250 >> >> I am in no doubt about general inadequacies of C et al=85 >> >> What I hate is an idea of setters and getters for my bitfields. With = >> pragma, compiler generated load/store will behave as needed. Everything = >> transparent and just-working. > > Until you have some data that is PDP-endian :-) > > Or want to split a field across a boundary that you're not supposed to > be able to split a field across. > > Or have a field that's semantically scattered/gathered. > > Can your proposed scheme represent IEEE floating point? > > I agree it is a very useful idea, I agree it has to be done "right". > But my definition of doing it "right" would be so demanding you'd be > crazy to do it in the compiler.... > > Mika > >> >> -- >> Divided by a common language >> >> Dragi=9Aa Duri=E6 >> dragisha at m3w.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Sun Aug 26 09:05:57 2012 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 Aug 2012 09:05:57 +0200 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> References: <180144C0-BAB7-4A97-8905-100EDF570167@m3w.org> <20120824181035.4EEFE1A207D@async.async.caltech.edu> <20120825191317.GA3747@zoho.com> <3939765F-96E5-4BB9-99FD-199C278D3729@m3w.org> <20120826045659.GA27610@zoho.com> Message-ID: <634D1148-0033-4037-A20E-AAAD05F0CFF1@m3w.org> Books don't disappear, usually :). They go out of print and move to afterlife in antiquity shops and forgotten shelves :). Google for table of contest of SPwM3 and match with names of SRC reports. I don't have a book with me right now so I can't write it out for you, sorry. -- Divided by a common language Dragi?a Duri? dragisha at m3w.org On Aug 26, 2012, at 6:56 AM, microcode at zoho.com wrote: > On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >> >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers) >> > > Oh thanks. I'll try to get a copy of that if it is good. I did download most > of the SRC stuff but have no idea when I'll have time to look at it. > > I'm trying to get a few books before they all disappear. > > -- > _ _ > ._ _ _ <_> ___ _ _ ___ ___ ___ _| | ___ > | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> > |_|_|_||_|\_|_.|_| \___/\_|_.\___/\___|\___. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jay.krell at cornell.edu Mon Aug 27 04:18:29 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 02:18:29 +0000 Subject: [M3devel] multiple evaluation hazard in backend? Message-ID: Tony, does m3front tend or tend not to ask the backend to evaluate the same expression multiple times, or does it tend to allocate a temporary if it would do so? It's just that my current approach, if not given temporaries, may tend to evaluate expressions multiple times that either must not (for correctness) or ought not (for efficiency). I guess I'll learn soon enough. My current "stack" is just TEXTs. Very simple. If I had to eliminate multiple evaluations I'd have to consider a few alternatives: 1) for every, push, do something like "{ T s0 = expr;" for every pop "}" That would require I always "use before pop", not "use not after pop" as I tend do. That isn't always easy. The above is also wrong. Instead of "s0", it'd be "sN" where N is the current stack depth, then "get(n)" would, like, get N-n. 2) Alternatively, if I knew the maximum stack depth such as by making two passes, I could have an array of unions. That would make casts less verbose -- they'd be via union member access, if no size change. 2b) I could maintain a stack pointer at compile time or runtime. Runtime easier but slower. I still might want to record types with my texts, like to avoid extra casts. Or maybe m3front does that already? Anyway, for now I can ignore it. I'm just starting in on function calls, and I still haven't handled nested functions. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 04:37:31 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 03:37:31 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120826045659.GA27610@zoho.com> Message-ID: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email and I'll tell you url) in a public site, SPwM3 is almost but last chapter is presumably in other hands (the so-called lost video tapes), and the other two books are (OK are not totally on-line, but bigger part are available from archive.org). I was trying hard to compile SPwM3 in one file, but the more I see it, I guess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was already available, we have already the C.1 - Introduction, C. 2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An? Introduction to programming with Threads (available from MS-Research in C# syntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are almost in source tree, it will just miss C. 5 - Thread Synchronization (but is one of the smallest < 20 p. I believe we can make a revised copy based on current Interface specification after we get a copy of ESC/M3). It needs a bit of work I guess, but if some kind soul is wanting to do this I will let him do it. The only needed book is Algorithms in Modula-3 (but there is a German script of very good quality from U Aachen) so, it's not too bad but sure will be nice to recover it from somewhere (I heard it was used to develop complete Algorithm Animations, which would be perfect for Mentor) Thanks in advance --- El s?b, 25/8/12, microcode at zoho.com escribi?: De: microcode at zoho.com Asunto: Re: [M3devel] Modula-3 bitfields Para: m3devel at elegosoft.com Fecha: s?bado, 25 de agosto, 2012 23:56 On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: > Systems Programming with Modula 3 > Edited by Greg Nelson > > Modula-3 bible. Most of material is available online in various SRC > reports (now on HP's ftp servers) > Oh thanks. I'll try to get a copy of that if it is good. I did download most of the SRC stuff but have no idea when I'll have time to look at it. I'm trying to get a few books before they all disappear. -- ? ? ? ? _? ? ? ? ? ? ? ? ? ? ? ? ? ???_? ? ? ._ _ _ <_> ___? _ _? ___? ___? ___? _| | ___ | ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> |_|_|_||_|\_|_.|_|? \___/\_|_.\___/\___|\___. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 05:08:31 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 03:08:31 +0000 Subject: [M3devel] race conditions in lock-free code... Message-ID: We have race conditions, like this, m3core/src/Text.m3: VAR fromCharCache := ARRAY CHAR OF T {NIL, ..}; (* 1-char texts *) PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; fromCharCache[c] := Text8.New (buf); END; RETURN fromCharCache [c] END FromChar; It should be: PROCEDURE FromChar (c: CHAR): T = VAR buf: ARRAY [0..0] OF CHAR; BEGIN IF fromCharCache [c] = NIL THEN buf [0] := c; WITH a = Text8.New (buf) DO MemoryBarrier(); fromCharCache[c] := a; END; END; RETURN fromCharCache [c] END FromChar; to ensure that all of Text8.New() finishes before the assignment to fromCharCache[c] is made. Can the compiler somehow catch these? I fear they are a small epidemic. For a long time people didn't realize where all the compiler and processor could reorder. Do we have the right constructs by now to fix them? I think we do. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 06:19:15 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:19:15 -0700 Subject: [M3devel] Modula-3 bitfields In-Reply-To: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> References: <1346035051.86854.YahooMailClassic@web29706.mail.ird.yahoo.com> Message-ID: <20120827041915.569CD1A207D@async.async.caltech.edu> If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3. The Handbook of Programming Languages has a chapter on Modula-3 as well. Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- From mika at async.caltech.edu Mon Aug 27 06:23:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 21:23:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: Message-ID: <20120827042351.141B81A207D@async.async.caltech.edu> Yeah it's a race condition, in theory. But is it important? If you get the wrong copy, there will be two TEXTs representing one CHAR. But no one ever said FromChar was guaranteed to return the same pointer when you call it with the same CHAR... Mika Jay K writes: >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >We have race conditions=2C like this=2C m3core/src/Text.m3: > > >VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > > >PROCEDURE FromChar (c: CHAR): T =3D > VAR buf: ARRAY [0..0] OF CHAR=3B > BEGIN > IF fromCharCache [c] =3D NIL THEN > buf [0] :=3D c=3B > fromCharCache[c] :=3D Text8.New (buf)=3B > END=3B > RETURN fromCharCache [c] > END FromChar=3B > > >It should be: > > >PROCEDURE FromChar (c: CHAR): T =3D >=0A= > VAR buf: ARRAY [0..0] OF CHAR=3B >=0A= > BEGIN >=0A= > IF fromCharCache [c] =3D NIL THEN >=0A= > buf [0] :=3D c=3B >=0A= > WITH a =3D Text8.New (buf) DO > MemoryBarrier()=3B > fromCharCache[c] :=3D a=3B > END=3B >=0A= > END=3B >=0A= > RETURN fromCharCache [c] >=0A= > END FromChar=3B >=0A= > > >to ensure that all of Text8.New() finishes before the assignment to fromCha= >rCache[c] is made. > > >Can the compiler somehow catch these? >I fear they are a small epidemic. >For a long time people didn't realize where all the compiler and processor = >could reorder. > > >Do we have the right constructs by now to fix them? >I think we do. > > > - Jay > = > >--_5f23896e-68ee-44da-82dd-311cd8c58979_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
We have race conditions=2C like = >this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >r (c: CHAR): T =3D
=0A= > =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > =3B BEGIN
=0A= > =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > =3B =3B =3B END=3B
=0A= > =3B =3B =3B RETURN fromCharCache [c]
=0A= > =3B END FromChar=3B
=0A= >

to ensure that all of Text8.New() finishes before the assignment to= > fromCharCache[c] is made.


Can the compiler somehow catch these?= >
I fear they are a small epidemic.
For a long time people didn't real= >ize where all the compiler and processor could reorder.


Do we ha= >ve the right constructs by now to fix them?
I think we do.


-= > Jay
>= > >--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 06:44:21 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 04:44:21 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 07:32:48 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 26 Aug 2012 22:32:48 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827042351.141B81A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> Message-ID: <20120827053248.A1B3F1A207D@async.async.caltech.edu> Sorry, I can't seem to quote your message. You say that Text8.New's result can be assigned to fromCharCache[c] before Text8.New is done running. How is that possible? Surely Text8.New has to calculate its result before it can be assigned anywhere. Yes possibly FromChar allocates space for the result of Text8.New but the assignment of whatever appears in that space can't happen until *after* Text8.New has run. Unless you are saying that the running of Text8.New and FromChar are intertwined somehow..? I know compilers do some odd things but that doesn't sound right... If Text8.New took fromCharCache[c] as a VAR parameter you might be right, though. But it doesn't... Am I missing something? Mika Mika Nystrom writes: > >Yeah it's a race condition, in theory. But is it important? If you get >the wrong copy, there will be two TEXTs representing one CHAR. But no >one ever said FromChar was guaranteed to return the same pointer when >you call it with the same CHAR... > > Mika > >Jay K writes: >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >>We have race conditions=2C like this=2C m3core/src/Text.m3: >> >> >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >> VAR buf: ARRAY [0..0] OF CHAR=3B >> BEGIN >> IF fromCharCache [c] =3D NIL THEN >> buf [0] :=3D c=3B >> fromCharCache[c] :=3D Text8.New (buf)=3B >> END=3B >> RETURN fromCharCache [c] >> END FromChar=3B >> >> >>It should be: >> >> >>PROCEDURE FromChar (c: CHAR): T =3D >>=0A= >> VAR buf: ARRAY [0..0] OF CHAR=3B >>=0A= >> BEGIN >>=0A= >> IF fromCharCache [c] =3D NIL THEN >>=0A= >> buf [0] :=3D c=3B >>=0A= >> WITH a =3D Text8.New (buf) DO >> MemoryBarrier()=3B >> fromCharCache[c] :=3D a=3B >> END=3B >>=0A= >> END=3B >>=0A= >> RETURN fromCharCache [c] >>=0A= >> END FromChar=3B >>=0A= >> >> >>to ensure that all of Text8.New() finishes before the assignment to fromCha= >>rCache[c] is made. >> >> >>Can the compiler somehow catch these? >>I fear they are a small epidemic. >>For a long time people didn't realize where all the compiler and processor = >>could reorder. >> >> >>Do we have the right constructs by now to fix them? >>I think we do. >> >> >> - Jay >> = >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
We have race conditions=2C like = >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= >>r (c: CHAR): T =3D
=0A= >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= >> =3B BEGIN
=0A= >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= >> =3B =3B =3B END=3B
=0A= >> =3B =3B =3B RETURN fromCharCache [c]
=0A= >> =3B END FromChar=3B
=0A= >>

to ensure that all of Text8.New() finishes before the assignment to= >> fromCharCache[c] is made.


Can the compiler somehow catch these?= >>
I fear they are a small epidemic.
For a long time people didn't real= >>ize where all the compiler and processor could reorder.


Do we ha= >>ve the right constructs by now to fix them?
I think we do.


-= >> Jay
>>= >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- From jay.krell at cornell.edu Mon Aug 27 08:27:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 06:27:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827053248.A1B3F1A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: The processor can reorder. Across call/ret at least in general. The compiler can also reorder, if you have whole-program-optimization, link-time-code-gen (Microsoft "LTCG"), link-time-optimization (gcc "LTO"). (We don't, but real world systems certainly do.) - Jay > To: jay.krell at cornell.edu; m3devel at elegosoft.com; mika at async.caltech.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Sun, 26 Aug 2012 22:32:48 -0700 > From: mika at async.caltech.edu > > Sorry, I can't seem to quote your message. > > You say that Text8.New's result can be assigned to fromCharCache[c] > before Text8.New is done running. How is that possible? Surely > Text8.New has to calculate its result before it can be assigned > anywhere. Yes possibly FromChar allocates space for the result of > Text8.New but the assignment of whatever appears in that space can't > happen until *after* Text8.New has run. Unless you are saying that > the running of Text8.New and FromChar are intertwined somehow..? > I know compilers do some odd things but that doesn't sound right... > > If Text8.New took fromCharCache[c] as a VAR parameter you might be > right, though. But it doesn't... > > Am I missing something? > > Mika > > Mika Nystrom writes: > > > >Yeah it's a race condition, in theory. But is it important? If you get > >the wrong copy, there will be two TEXTs representing one CHAR. But no > >one ever said FromChar was guaranteed to return the same pointer when > >you call it with the same CHAR... > > > > Mika > > > >Jay K writes: > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/plain; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >>We have race conditions=2C like this=2C m3core/src/Text.m3: > >> > >> > >>VAR fromCharCache :=3D ARRAY CHAR OF T {NIL=2C ..}=3B (* 1-char texts *) > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >> BEGIN > >> IF fromCharCache [c] =3D NIL THEN > >> buf [0] :=3D c=3B > >> fromCharCache[c] :=3D Text8.New (buf)=3B > >> END=3B > >> RETURN fromCharCache [c] > >> END FromChar=3B > >> > >> > >>It should be: > >> > >> > >>PROCEDURE FromChar (c: CHAR): T =3D > >>=0A= > >> VAR buf: ARRAY [0..0] OF CHAR=3B > >>=0A= > >> BEGIN > >>=0A= > >> IF fromCharCache [c] =3D NIL THEN > >>=0A= > >> buf [0] :=3D c=3B > >>=0A= > >> WITH a =3D Text8.New (buf) DO > >> MemoryBarrier()=3B > >> fromCharCache[c] :=3D a=3B > >> END=3B > >>=0A= > >> END=3B > >>=0A= > >> RETURN fromCharCache [c] > >>=0A= > >> END FromChar=3B > >>=0A= > >> > >> > >>to ensure that all of Text8.New() finishes before the assignment to fromCha= > >>rCache[c] is made. > >> > >> > >>Can the compiler somehow catch these? > >>I fear they are a small epidemic. > >>For a long time people didn't realize where all the compiler and processor = > >>could reorder. > >> > >> > >>Do we have the right constructs by now to fix them? > >>I think we do. > >> > >> > >> - Jay > >> = > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>Content-Type: text/html; charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
We have race conditions=2C like = > >>this=2C m3core/src/Text.m3:


VAR fromCharCache :=3D ARRAY CHAR OF= > >> T {NIL=2C ..}=3B (* 1-char texts *)


PROCEDURE FromChar (c: CHAR= > >>): T =3D
 =3B VAR buf: ARRAY [0..0] OF CHAR=3B
 =3B BEGIN
= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
 =3B = > >>=3B =3B =3B =3B buf [0] :=3D c=3B
 =3B =3B =3B&n= > >>bsp=3B =3B fromCharCache[c] :=3D Text8.New (buf)=3B
 =3B =3B= > >> =3B END=3B
 =3B =3B =3B RETURN fromCharCache [c]
&nb= > >>sp=3B END FromChar=3B


It should be:


PROCEDURE FromCha= > >>r (c: CHAR): T =3D
=0A= > >> =3B VAR buf: ARRAY [0..0] OF CHAR=3B
=0A= > >> =3B BEGIN
=0A= > >> =3B =3B =3B IF fromCharCache [c] =3D NIL THEN
=0A= > >> =3B =3B =3B =3B =3B buf [0] :=3D c=3B
=0A= > >> =3B =3B =3B =3B =3B WITH a =3D Text8.New (buf) DO
&= > >>nbsp=3B =3B =3B =3B =3B =3B =3B MemoryBarrier()=3B<= > >>br> =3B =3B =3B =3B =3B =3B =3B fromCharCache[c= > >>] :=3D a=3B
 =3B =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B END=3B
=0A= > >> =3B =3B =3B RETURN fromCharCache [c]
=0A= > >> =3B END FromChar=3B
=0A= > >>

to ensure that all of Text8.New() finishes before the assignment to= > >> fromCharCache[c] is made.


Can the compiler somehow catch these?= > >>
I fear they are a small epidemic.
For a long time people didn't real= > >>ize where all the compiler and processor could reorder.


Do we ha= > >>ve the right constructs by now to fix them?
I think we do.


-= > >> Jay
> >>= > >> > >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 09:17:44 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 00:17:44 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu>, <20120827053248.A1B3F1A207D@async.async.caltech.edu> Message-ID: <20120827071744.781621A207D@async.async.caltech.edu> Oh I see now. It's not that the processor reorders in any tricky way. It is that the write performed by Text8.New could be sitting around processor 1's cache indefinitely without actually getting written back to main memory. Processor 2 might see the new pointer, but when reading the memory pointed to, would get uninitialized memory. I think this can happen on Alpha but maybe not X86 (without further reorderings anyhow)? Any chance any of the pthreads bugs could be of this nature? How are the pthreads doing? I gathered some bugs were fixed but I haven't tried the system for a while. Mika Jay K writes: >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >The processor can reorder. Across call/ret at least in general. > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >. (We don't=2C but real world systems certainly do.) > > > - Jay > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >> From: mika at async.caltech.edu >>=20 >> Sorry=2C I can't seem to quote your message. >>=20 >> You say that Text8.New's result can be assigned to fromCharCache[c] >> before Text8.New is done running. How is that possible? Surely >> Text8.New has to calculate its result before it can be assigned >> anywhere. Yes possibly FromChar allocates space for the result of >> Text8.New but the assignment of whatever appears in that space can't >> happen until *after* Text8.New has run. Unless you are saying that >> the running of Text8.New and FromChar are intertwined somehow..? >> I know compilers do some odd things but that doesn't sound right... >>=20 >> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> right=2C though. But it doesn't... >>=20 >> Am I missing something? >>=20 >> Mika >>=20 >> Mika Nystrom writes: >> > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= >et >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no >> >one ever said FromChar was guaranteed to return the same pointer when >> >you call it with the same CHAR... >> > >> > Mika >> > >> >Jay K writes: >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >> >> >> >> >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >exts *) >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >> BEGIN >> >> IF fromCharCache [c] =3D3D NIL THEN >> >> buf [0] :=3D3D c=3D3B >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >> >> END=3D3B >> >> RETURN fromCharCache [c] >> >> END FromChar=3D3B >> >> >> >> >> >>It should be: >> >> >> >> >> >>PROCEDURE FromChar (c: CHAR): T =3D3D >> >>=3D0A=3D >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B >> >>=3D0A=3D >> >> BEGIN >> >>=3D0A=3D >> >> IF fromCharCache [c] =3D3D NIL THEN >> >>=3D0A=3D >> >> buf [0] :=3D3D c=3D3B >> >>=3D0A=3D >> >> WITH a =3D3D Text8.New (buf) DO >> >> MemoryBarrier()=3D3B >> >> fromCharCache[c] :=3D3D a=3D3B >> >> END=3D3B >> >>=3D0A=3D >> >> END=3D3B >> >>=3D0A=3D >> >> RETURN fromCharCache [c] >> >>=3D0A=3D >> >> END FromChar=3D3B >> >>=3D0A=3D >> >> >> >> >> >>to ensure that all of Text8.New() finishes before the assignment to fro= >mCha=3D >> >>rCache[c] is made. >> >> >> >> >> >>Can the compiler somehow catch these? >> >>I fear they are a small epidemic. >> >>For a long time people didn't realize where all the compiler and proces= >sor =3D >> >>could reorder. >> >> >> >> >> >>Do we have the right constructs by now to fix them? >> >>I think we do. >> >> >> >> >> >> - Jay >> >> =3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" >> >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >> >> >> >>
We have race conditions= >=3D2C like =3D >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > CHAR OF=3D >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >(c: CHAR=3D >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >3B BEGIN
=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >nbsp=3D3B =3D >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > =3D3B =3D3B&n=3D >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >p=3D3B =3D3B=3D >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >che [c]
&nb=3D >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > FromCha=3D >> >>r (c: CHAR): T =3D3D
=3D0A=3D >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >> >> =3D3B BEGIN
=3D0A=3D >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >B
=3D0A=3D >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >ew (buf) DO
&=3D >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >emoryBarrier()=3D3B<=3D >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >3B fromCharCache[c=3D >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>=3D0A=3D >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >> >> =3D3B END FromChar=3D3B
=3D0A=3D >> >>

to ensure that all of Text8.New() finishes before the assignmen= >t to=3D >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= >ese?=3D >> >>
I fear they are a small epidemic.
For a long time people didn't = >real=3D >> >>ize where all the compiler and processor could reorder.


Do w= >e ha=3D >> >>ve the right constructs by now to fix them?
I think we do.

r> -=3D >> >> Jay
>> >>=3D >> >> >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > = > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
The processor can reorder. Acros= >s call/ret at least in general.


The compiler can also reorder=2C= > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >stems certainly do.)


 =3B- Jay


yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >is done running. How is that possible? Surely
>=3B Text8.New has to = >calculate its result before it can be assigned
>=3B anywhere. Yes pos= >sibly FromChar allocates space for the result of
>=3B Text8.New but th= >e assignment of whatever appears in that space can't
>=3B happen until= > *after* Text8.New has run. Unless you are saying that
>=3B the runni= >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >mpilers do some odd things but that doesn't sound right...
>=3B
&g= >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >one ever said FromChar was guaranteed to return the same pointer when
&g= >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >=3B >=3B>=3BFor a long time people didn't realize where all the compile= >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >-311cd8c58979_--
>= > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 10:55:09 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:55:09 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 27 10:58:48 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 08:58:48 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: , <20120827042351.141B81A207D@async.async.caltech.edu>, , <20120827053248.A1B3F1A207D@async.async.caltech.edu>, , <20120827071744.781621A207D@async.async.caltech.edu>, Message-ID: Also, a compiler is free to reorder this stuff as well. If it can see the body of the function and inline it. Without an explicit lock/fence/barrier of some kind, including a call to a function that the compiler cannot see the body of, the compiler can reorder. Our compiler doesn't see far, so that helps. But that is an implementation detail of our compiler. The C backend will make "WPO" / "LTCG" / "LTO" sort of more viable..and the compiler will then see more. (all names for the same thing -- whole program optimization, link time code generation, link time optimization) - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] race conditions in lock-free code... Date: Mon, 27 Aug 2012 08:55:09 +0000 Sorry, right, it is about cache not processor. A uniprocessor system has no race. Uniprocessor systems barely exist any longer. I wouldn't put this reordering past any processor, including x86. But I don't know. It is unlikely this is a/the problem with pthreads, but I don't know. I know at least one major problem with pthreads was fixed. I haven't been looking at it. I was meaning to check if the pthread bug that was fixed applied to Win32. I'm deep in a C backend right now..really really hoping to pull it off... - Jay > To: jay.krell at cornell.edu > Date: Mon, 27 Aug 2012 00:17:44 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] race conditions in lock-free code... > > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > >The processor can reorder. Across call/ret at least in general. > > > > > >The compiler can also reorder=2C if you have whole-program-optimization=2C = > >link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >. (We don't=2C but real world systems certainly do.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >edu > >> Subject: Re: [M3devel] race conditions in lock-free code... > >> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >> From: mika at async.caltech.edu > >>=20 > >> Sorry=2C I can't seem to quote your message. > >>=20 > >> You say that Text8.New's result can be assigned to fromCharCache[c] > >> before Text8.New is done running. How is that possible? Surely > >> Text8.New has to calculate its result before it can be assigned > >> anywhere. Yes possibly FromChar allocates space for the result of > >> Text8.New but the assignment of whatever appears in that space can't > >> happen until *after* Text8.New has run. Unless you are saying that > >> the running of Text8.New and FromChar are intertwined somehow..? > >> I know compilers do some odd things but that doesn't sound right... > >>=20 > >> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >> right=2C though. But it doesn't... > >>=20 > >> Am I missing something? > >>=20 > >> Mika > >>=20 > >> Mika Nystrom writes: > >> > > >> >Yeah it's a race condition=2C in theory. But is it important? If you g= > >et > >> >the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >> >one ever said FromChar was guaranteed to return the same pointer when > >> >you call it with the same CHAR... > >> > > >> > Mika > >> > > >> >Jay K writes: > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >>We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >> >> > >> >> > >> >>VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >exts *) > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >> BEGIN > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >> buf [0] :=3D3D c=3D3B > >> >> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >> >> END=3D3B > >> >> RETURN fromCharCache [c] > >> >> END FromChar=3D3B > >> >> > >> >> > >> >>It should be: > >> >> > >> >> > >> >>PROCEDURE FromChar (c: CHAR): T =3D3D > >> >>=3D0A=3D > >> >> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >> >>=3D0A=3D > >> >> BEGIN > >> >>=3D0A=3D > >> >> IF fromCharCache [c] =3D3D NIL THEN > >> >>=3D0A=3D > >> >> buf [0] :=3D3D c=3D3B > >> >>=3D0A=3D > >> >> WITH a =3D3D Text8.New (buf) DO > >> >> MemoryBarrier()=3D3B > >> >> fromCharCache[c] :=3D3D a=3D3B > >> >> END=3D3B > >> >>=3D0A=3D > >> >> END=3D3B > >> >>=3D0A=3D > >> >> RETURN fromCharCache [c] > >> >>=3D0A=3D > >> >> END FromChar=3D3B > >> >>=3D0A=3D > >> >> > >> >> > >> >>to ensure that all of Text8.New() finishes before the assignment to fro= > >mCha=3D > >> >>rCache[c] is made. > >> >> > >> >> > >> >>Can the compiler somehow catch these? > >> >>I fear they are a small epidemic. > >> >>For a long time people didn't realize where all the compiler and proces= > >sor =3D > >> >>could reorder. > >> >> > >> >> > >> >>Do we have the right constructs by now to fix them? > >> >>I think we do. > >> >> > >> >> > >> >> - Jay > >> >> =3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_ > >> >>Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >>Content-Transfer-Encoding: quoted-printable > >> >> > >> >> > >> >> > >> >> > >> >>
We have race conditions= > >=3D2C like =3D > >> >>this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > > CHAR OF=3D > >> >> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >(c: CHAR=3D > >> >>): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >3B BEGIN
=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >nbsp=3D3B =3D > >> >>=3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > > =3D3B =3D3B&n=3D > >> >>bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >p=3D3B =3D3B=3D > >> >> =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >che [c]
&nb=3D > >> >>sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > > FromCha=3D > >> >>r (c: CHAR): T =3D3D
=3D0A=3D > >> >> =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >> >> =3D3B BEGIN
=3D0A=3D > >> >> =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >ew (buf) DO
&=3D > >> >>nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >emoryBarrier()=3D3B<=3D > >> >>br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >3B fromCharCache[c=3D > >> >>] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>=3D0A=3D > >> >> =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >> >> =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >> >> =3D3B END FromChar=3D3B
=3D0A=3D > >> >>

to ensure that all of Text8.New() finishes before the assignmen= > >t to=3D > >> >> fromCharCache[c] is made.


Can the compiler somehow catch th= > >ese?=3D > >> >>
I fear they are a small epidemic.
For a long time people didn't = > >real=3D > >> >>ize where all the compiler and processor could reorder.


Do w= > >e ha=3D > >> >>ve the right constructs by now to fix them?
I think we do.

>r> -=3D > >> >> Jay
> >> >>=3D > >> >> > >> >>--_5f23896e-68ee-44da-82dd-311cd8c58979_-- > > = > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > >
The processor can reorder. Acros= > >s call/ret at least in general.


The compiler can also reorder=2C= > > if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >stems certainly do.)


 =3B- Jay


>yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >is done running. How is that possible? Surely
>=3B Text8.New has to = > >calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >e assignment of whatever appears in that space can't
>=3B happen until= > > *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >>=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >=2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >one ever said FromChar was guaranteed to return the same pointer when
&g= > >t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >=3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >=3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >=3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > > IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >=3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >=3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >=3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >=3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > > to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >=3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >=3B >=3B>=3BFor a long time people didn't realize where all the compile= > >r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >=3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > ><=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >=3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >=3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >=3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >-->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > > >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > > OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >[0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >=3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >=3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > > ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >=3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >=3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >=3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > > where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >=3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > ><=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >=3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >-311cd8c58979_--
> >= > > > >--_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Mon Aug 27 14:20:56 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 13:20:56 +0100 (BST) Subject: [M3devel] Modula-3 bitfields In-Reply-To: <20120827041915.569CD1A207D@async.async.caltech.edu> Message-ID: <1346070056.46764.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: Yet, put another "A Theory of Objects" is almost begin to end in the web, however it misses just the justice with "Baby Modula-3" as it was used to check the type system of Modula-3, I believe but this research reports never came out (or the library didn't published it) so it's difficult to know what else is missing. In fact they wrote an accompanying software to it, a new implementation of Obliq based on first chapters of the book, don't know where it ended. Thanks --- El dom, 26/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Modula-3 bitfields Para: "Daniel Alejandro Benavides D." CC: m3devel at elegosoft.com Fecha: domingo, 26 de agosto, 2012 23:19 If you're trying to build the "complete Modula-3 library", Sedgewick also did an edition of his Algorithms in Modula-3.? The Handbook of Programming Languages has a chapter on Modula-3 as well. ? ???Mika "Daniel Alejandro Benavides D." writes: >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/plain; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >Hi all: >Yeah, but don't worry Modula-3 by Harbison is on-line (write me an email an= >d I'll tell you url) in a public site, SPwM3 is almost but last chapter is = >presumably in other hands (the so-called lost video tapes), and the other t= >wo books are (OK are not totally on-line, but bigger part are available fro= >m archive.org). >I was trying hard to compile SPwM3 in one file, but the more I see it, I gu= >ess we need to remake it from tex, to avoid non-ocr parts, as Greg Nelson s= >aid that was already available, we have already the C.1 - Introduction, C. = >2 -Language Definition, and need C. 3 - Standard interfaces, C. 4 - An=A0 I= >ntroduction to programming with Threads (available from MS-Research in C# s= >yntax but everything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tuto= >rial, are almost in source tree, it will just miss C. 5 - Thread Synchroniz= >ation (but is one of the smallest < 20 p. I believe we can make a revised c= >opy based on current Interface specification after we get a copy of ESC/M3)= >. > >It needs a bit of work I guess, but if some kind soul is wanting to do this= > I will let him do it. >The only needed book is Algorithms in Modula-3 (but there is a German scrip= >t of very good quality from U Aachen) so, it's not too bad but sure will be= > nice to recover it from somewhere (I heard it was used to develop complete= > Algorithm Animations, which would be perfect for Mentor) > >Thanks in advance > >--- El s=E1b, 25/8/12, microcode at zoho.com escribi=F3: > >De: microcode at zoho.com >Asunto: Re: [M3devel] Modula-3 bitfields >Para: m3devel at elegosoft.com >Fecha: s=E1bado, 25 de agosto, 2012 23:56 > >On Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote: >> Systems Programming with Modula 3 >> Edited by Greg Nelson >>=20 >> Modula-3 bible. Most of material is available online in various SRC >> reports (now on HP's ftp servers)=20 >>=20 > >Oh thanks. I'll try to get a copy of that if it is good. I did download mos= >t >of the SRC stuff but have no idea when I'll have time to look at it. > >I'm trying to get a few books before they all disappear. > >--=20 >=A0 =A0 =A0 =A0 _=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0= >=A0_=A0 =A0 =A0=20 >._ _ _ <_> ___=A0 _ _=A0 ___=A0 ___=A0 ___=A0 _| | ___=20 >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._> >|_|_|_||_|\_|_.|_|=A0 \___/\_|_.\___/\___|\___. >=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= > =A0 =A0 =A0=A0=A0 > > >---2131580713-1558587311-1346035051=:86854 >Content-Type: text/html; charset=iso-8859-1 >Content-Transfer-Encoding: quoted-printable > >
top" style=3D"font: inherit;">Hi all:
Yeah, but don't worry Modula-3 by = >Harbison is on-line (write me an email and I'll tell you url) in a public s= >ite, SPwM3 is almost but last chapter is presumably in other hands (the so-= >called lost video tapes), and the other two books are (OK are not totally o= >n-line, but bigger part are available from archive.org).
I was trying ha= >rd to compile SPwM3 in one file, but the more I see it, I guess we need to = >remake it from tex, to avoid non-ocr parts, as Greg Nelson said that was al= >ready available, we have already the C.1 - Introduction, C. 2 -Language Def= >inition, and need C. 3 - Standard interfaces, C. 4 - An  Introduction = >to programming with Threads (available from MS-Research in C# syntax but ev= >erything else is same), C. 6 - I/O Streams, C. 7 - Trestle Tutorial, are al= >most in source tree, it will just miss C. 5 - Thread Synchronization (but i= >s > one of the smallest < 20 p. I believe we can make a revised copy based = >on current Interface specification after we get a copy of ESC/M3).

I= >t needs a bit of work I guess, but if some kind soul is wanting to do this = >I will let him do it.
The only needed book is Algorithms in Modula-3 (bu= >t there is a German script of very good quality from U Aachen) so, it's not= > too bad but sure will be nice to recover it from somewhere (I heard it was= > used to develop complete Algorithm Animations, which would be perfect for = >Mentor)

Thanks in advance

--- El s=E1b, 25/8/12, microcode= >@zoho.com <microcode at zoho.com> escribi=F3:
style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding= >-left: 5px;">
De: microcode at zoho.com <microcode at zoho.com>
Asunt= >o: Re: [M3devel] Modula-3 bitfields
Para: m3devel at elegosoft.com
Fecha= >: s=E1bado, 25 de agosto, 2012 23:56

On > Sat, Aug 25, 2012 at 10:08:46PM +0200, Dragi??a Duri?? wrote:
> Syst= >ems Programming with Modula 3
> Edited by Greg Nelson
>
>= >; Modula-3 bible. Most of material is available online in various SRC
&g= >t; reports (now on HP's ftp servers)
>

Oh thanks. I'll try t= >o get a copy of that if it is good. I did download most
of the SRC stuff= > but have no idea when I'll have time to look at it.

I'm trying to g= >et a few books before they all disappear.

--
     = >;   _                  &n= >bsp;          _     
._ _ = >_ <_> ___  _ _  ___  ___  ___  _| | ___
= >| ' ' || |/ | '| '_>/ . \/ | '/ . \/ . |/ ._>
|_|_|_||_|\_|_.|_|&n= >bsp; \___/\_|_.\___/\___|\___.
           = >                   >                

= >
>---2131580713-1558587311-1346035051=:86854-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Aug 27 15:27:38 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 09:27:38 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827071744.781621A207D@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> Message-ID: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. Sent from my iPad On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > Oh I see now. > > It's not that the processor reorders in any tricky way. It is that the > write performed by Text8.New could be sitting around processor 1's cache > indefinitely without actually getting written back to main memory. > Processor 2 might see the new pointer, but when reading the memory pointed > to, would get uninitialized memory. I think this can happen on Alpha but > maybe not X86 (without further reorderings anyhow)? > > Any chance any of the pthreads bugs could be of this nature? How are the > pthreads doing? I gathered some bugs were fixed but I haven't tried the > system for a while. > > Mika > > > Jay K writes: >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> The processor can reorder. Across call/ret at least in general. >> >> >> The compiler can also reorder=2C if you have whole-program-optimization=2C = >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= >> . (We don't=2C but real world systems certainly do.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= >> edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 >>> From: mika at async.caltech.edu >>> =20 >>> Sorry=2C I can't seem to quote your message. >>> =20 >>> You say that Text8.New's result can be assigned to fromCharCache[c] >>> before Text8.New is done running. How is that possible? Surely >>> Text8.New has to calculate its result before it can be assigned >>> anywhere. Yes possibly FromChar allocates space for the result of >>> Text8.New but the assignment of whatever appears in that space can't >>> happen until *after* Text8.New has run. Unless you are saying that >>> the running of Text8.New and FromChar are intertwined somehow..? >>> I know compilers do some odd things but that doesn't sound right... >>> =20 >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>> right=2C though. But it doesn't... >>> =20 >>> Am I missing something? >>> =20 >>> Mika >>> =20 >>> Mika Nystrom writes: >>>> >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= >> et >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no >>>> one ever said FromChar was guaranteed to return the same pointer when >>>> you call it with the same CHAR... >>>> >>>> Mika >>>> >>>> Jay K writes: >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>>>> >>>>> >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= >> exts *) >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> BEGIN >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> buf [0] :=3D3D c=3D3B >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B >>>>> END=3D3B >>>>> RETURN fromCharCache [c] >>>>> END FromChar=3D3B >>>>> >>>>> >>>>> It should be: >>>>> >>>>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D >>>>> =3D0A=3D >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B >>>>> =3D0A=3D >>>>> BEGIN >>>>> =3D0A=3D >>>>> IF fromCharCache [c] =3D3D NIL THEN >>>>> =3D0A=3D >>>>> buf [0] :=3D3D c=3D3B >>>>> =3D0A=3D >>>>> WITH a =3D3D Text8.New (buf) DO >>>>> MemoryBarrier()=3D3B >>>>> fromCharCache[c] :=3D3D a=3D3B >>>>> END=3D3B >>>>> =3D0A=3D >>>>> END=3D3B >>>>> =3D0A=3D >>>>> RETURN fromCharCache [c] >>>>> =3D0A=3D >>>>> END FromChar=3D3B >>>>> =3D0A=3D >>>>> >>>>> >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= >> mCha=3D >>>>> rCache[c] is made. >>>>> >>>>> >>>>> Can the compiler somehow catch these? >>>>> I fear they are a small epidemic. >>>>> For a long time people didn't realize where all the compiler and proces= >> sor =3D >>>>> could reorder. >>>>> >>>>> >>>>> Do we have the right constructs by now to fix them? >>>>> I think we do. >>>>> >>>>> >>>>> - Jay >>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> >>>>> >>>>> >>>>>
We have race conditions= >> =3D2C like =3D >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= >> CHAR OF=3D >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = >> (c: CHAR=3D >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= >> 3B BEGIN
=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= >> nbsp=3D3B =3D >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= >>  =3D3B =3D3B&n=3D >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= >> p=3D3B =3D3B=3D >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= >> che [c]
&nb=3D >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= >> FromCha=3D >>>>> r (c: CHAR): T =3D3D
=3D0A=3D >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D >>>>>  =3D3B BEGIN
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= >> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= >> B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= >> ew (buf) DO
&=3D >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= >> emoryBarrier()=3D3B<=3D >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= >> 3B fromCharCache[c=3D >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B>> =3D0A=3D >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D >>>>>

to ensure that all of Text8.New() finishes before the assignmen= >> t to=3D >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= >> ese?=3D >>>>>
I fear they are a small epidemic.
For a long time people didn't = >> real=3D >>>>> ize where all the compiler and processor could reorder.


Do w= >> e ha=3D >>>>> ve the right constructs by now to fix them?
I think we do.

> r> -=3D >>>>> Jay
>>>>> =3D >>>>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
The processor can reorder. Acros= >> s call/ret at least in general.


The compiler can also reorder=2C= >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= >> stems certainly do.)


 =3B- Jay


> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = >> is done running. How is that possible? Surely
>=3B Text8.New has to = >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= >> e assignment of whatever appears in that space can't
>=3B happen until= >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= >> one ever said FromChar was guaranteed to return the same pointer when
&g= >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3:>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= >> -311cd8c58979_--
>> = >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- From jay.krell at cornell.edu Mon Aug 27 20:07:10 2012 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Aug 2012 18:07:10 +0000 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: It is NOT benign. The global can be written before the initialization of what it points to is done. The result is wrong. It IS benign if multiple threads go and allocate separate multiple initialized TEXTs. That's not the problem. - Jay > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > From: hosking at cs.purdue.edu > Subject: Re: [M3devel] race conditions in lock-free code... > Date: Mon, 27 Aug 2012 09:27:38 -0400 > To: mika at async.caltech.edu > > So why is this anything other than benign? It's a cache and there's no problem if different threads get different texts. Don't cripple performance unnecessarily. The collector will reclaim any unreachable text. > > Sent from my iPad > > On Aug 27, 2012, at 3:17 AM, Mika Nystrom wrote: > > > Oh I see now. > > > > It's not that the processor reorders in any tricky way. It is that the > > write performed by Text8.New could be sitting around processor 1's cache > > indefinitely without actually getting written back to main memory. > > Processor 2 might see the new pointer, but when reading the memory pointed > > to, would get uninitialized memory. I think this can happen on Alpha but > > maybe not X86 (without further reorderings anyhow)? > > > > Any chance any of the pthreads bugs could be of this nature? How are the > > pthreads doing? I gathered some bugs were fixed but I haven't tried the > > system for a while. > > > > Mika > > > > > > Jay K writes: > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> The processor can reorder. Across call/ret at least in general. > >> > >> > >> The compiler can also reorder=2C if you have whole-program-optimization=2C = > >> link-time-code-gen (Microsoft "LTCG")=2C link-time-optimization (gcc "LTO")= > >> . (We don't=2C but real world systems certainly do.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu=3B m3devel at elegosoft.com=3B mika at async.caltech.= > >> edu > >>> Subject: Re: [M3devel] race conditions in lock-free code... > >>> Date: Sun=2C 26 Aug 2012 22:32:48 -0700 > >>> From: mika at async.caltech.edu > >>> =20 > >>> Sorry=2C I can't seem to quote your message. > >>> =20 > >>> You say that Text8.New's result can be assigned to fromCharCache[c] > >>> before Text8.New is done running. How is that possible? Surely > >>> Text8.New has to calculate its result before it can be assigned > >>> anywhere. Yes possibly FromChar allocates space for the result of > >>> Text8.New but the assignment of whatever appears in that space can't > >>> happen until *after* Text8.New has run. Unless you are saying that > >>> the running of Text8.New and FromChar are intertwined somehow..? > >>> I know compilers do some odd things but that doesn't sound right... > >>> =20 > >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be > >>> right=2C though. But it doesn't... > >>> =20 > >>> Am I missing something? > >>> =20 > >>> Mika > >>> =20 > >>> Mika Nystrom writes: > >>>> > >>>> Yeah it's a race condition=2C in theory. But is it important? If you g= > >> et > >>>> the wrong copy=2C there will be two TEXTs representing one CHAR. But no > >>>> one ever said FromChar was guaranteed to return the same pointer when > >>>> you call it with the same CHAR... > >>>> > >>>> Mika > >>>> > >>>> Jay K writes: > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> We have race conditions=3D2C like this=3D2C m3core/src/Text.m3: > >>>>> > >>>>> > >>>>> VAR fromCharCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char t= > >> exts *) > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> BEGIN > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> buf [0] :=3D3D c=3D3B > >>>>> fromCharCache[c] :=3D3D Text8.New (buf)=3D3B > >>>>> END=3D3B > >>>>> RETURN fromCharCache [c] > >>>>> END FromChar=3D3B > >>>>> > >>>>> > >>>>> It should be: > >>>>> > >>>>> > >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D > >>>>> =3D0A=3D > >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3B > >>>>> =3D0A=3D > >>>>> BEGIN > >>>>> =3D0A=3D > >>>>> IF fromCharCache [c] =3D3D NIL THEN > >>>>> =3D0A=3D > >>>>> buf [0] :=3D3D c=3D3B > >>>>> =3D0A=3D > >>>>> WITH a =3D3D Text8.New (buf) DO > >>>>> MemoryBarrier()=3D3B > >>>>> fromCharCache[c] :=3D3D a=3D3B > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> END=3D3B > >>>>> =3D0A=3D > >>>>> RETURN fromCharCache [c] > >>>>> =3D0A=3D > >>>>> END FromChar=3D3B > >>>>> =3D0A=3D > >>>>> > >>>>> > >>>>> to ensure that all of Text8.New() finishes before the assignment to fro= > >> mCha=3D > >>>>> rCache[c] is made. > >>>>> > >>>>> > >>>>> Can the compiler somehow catch these? > >>>>> I fear they are a small epidemic. > >>>>> For a long time people didn't realize where all the compiler and proces= > >> sor =3D > >>>>> could reorder. > >>>>> > >>>>> > >>>>> Do we have the right constructs by now to fix them? > >>>>> I think we do. > >>>>> > >>>>> > >>>>> - Jay > >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ > >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>>> Content-Transfer-Encoding: quoted-printable > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>
We have race conditions= > >> =3D2C like =3D > >>>>> this=3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D ARRAY= > >> CHAR OF=3D > >>>>> T {NIL=3D2C ..}=3D3B (* 1-char texts *)


PROCEDURE FromChar = > >> (c: CHAR=3D > >>>>> ): T =3D3D
 =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
 =3D= > >> 3B BEGIN
=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
&= > >> nbsp=3D3B =3D > >>>>> =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3B
 =3D3B= > >>  =3D3B =3D3B&n=3D > >>>>> bsp=3D3B =3D3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B
&nbs= > >> p=3D3B =3D3B=3D > >>>>>  =3D3B END=3D3B
 =3D3B =3D3B =3D3B RETURN fromCharCa= > >> che [c]
&nb=3D > >>>>> sp=3D3B END FromChar=3D3B


It should be:


PROCEDURE= > >> FromCha=3D > >>>>> r (c: CHAR): T =3D3D
=3D0A=3D > >>>>>  =3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3B
=3D0A=3D > >>>>>  =3D3B BEGIN
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B IF fromCharCache [c] =3D3D NIL THEN
= > >> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B buf [0] :=3D3D c=3D3= > >> B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B =3D3B =3D3B WITH a =3D3D Text8.N= > >> ew (buf) DO
&=3D > >>>>> nbsp=3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B M= > >> emoryBarrier()=3D3B<=3D > >>>>> br> =3D3B =3D3B =3D3B =3D3B =3D3B =3D3B =3D= > >> 3B fromCharCache[c=3D > >>>>> ] :=3D3D a=3D3B
 =3D3B =3D3B =3D3B =3D3B END=3D3B >>> =3D0A=3D > >>>>>  =3D3B =3D3B =3D3B END=3D3B
=3D0A=3D > >>>>>  =3D3B =3D3B =3D3B RETURN fromCharCache [c]
=3D0A=3D > >>>>>  =3D3B END FromChar=3D3B
=3D0A=3D > >>>>>

to ensure that all of Text8.New() finishes before the assignmen= > >> t to=3D > >>>>> fromCharCache[c] is made.


Can the compiler somehow catch th= > >> ese?=3D > >>>>>
I fear they are a small epidemic.
For a long time people didn't = > >> real=3D > >>>>> ize where all the compiler and processor could reorder.


Do w= > >> e ha=3D > >>>>> ve the right constructs by now to fix them?
I think we do.

>> r> -=3D > >>>>> Jay
> >>>>> =3D > >>>>> > >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- > >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
The processor can reorder. Acros= > >> s call/ret at least in general.


The compiler can also reorder=2C= > >> if you have whole-program-optimization=2C link-time-code-gen (Microsoft "L= > >> TCG")=2C link-time-optimization (gcc "LTO"). (We don't=2C but real world sy= > >> stems certainly do.)


 =3B- Jay


>> yDrivePlaceholder">
>=3B To: jay.krell at cornell.edu=3B m3devel at elegos= > >> oft.com=3B mika at async.caltech.edu
>=3B Subject: Re: [M3devel] race con= > >> ditions in lock-free code...
>=3B Date: Sun=2C 26 Aug 2012 22:32:48 -0= > >> 700
>=3B From: mika at async.caltech.edu
>=3B
>=3B Sorry=2C I = > >> can't seem to quote your message.
>=3B
>=3B You say that Text8.N= > >> ew's result can be assigned to fromCharCache[c]
>=3B before Text8.New = > >> is done running. How is that possible? Surely
>=3B Text8.New has to = > >> calculate its result before it can be assigned
>=3B anywhere. Yes pos= > >> sibly FromChar allocates space for the result of
>=3B Text8.New but th= > >> e assignment of whatever appears in that space can't
>=3B happen until= > >> *after* Text8.New has run. Unless you are saying that
>=3B the runni= > >> ng of Text8.New and FromChar are intertwined somehow..?
>=3B I know co= > >> mpilers do some odd things but that doesn't sound right...
>=3B
&g= > >> t=3B If Text8.New took fromCharCache[c] as a VAR parameter you might be
= > >> >=3B right=2C though. But it doesn't...
>=3B
>=3B Am I missin= > >> g something?
>=3B
>=3B Mika
>=3B
>=3B Mika Nystr= > >> om writes:
>=3B >=3B
>=3B >=3BYeah it's a race condition=2C i= > >> n theory. But is it important? If you get
>=3B >=3Bthe wrong copy= > >> =2C there will be two TEXTs representing one CHAR. But no
>=3B >=3B= > >> one ever said FromChar was guaranteed to return the same pointer when
&g= > >> t=3B >=3Byou call it with the same CHAR...
>=3B >=3B
>=3B >= > >> =3B Mika
>=3B >=3B
>=3B >=3BJay K writes:
>=3B >= > >> =3B>=3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BCon= > >> tent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BConte= > >> nt-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B >= > >> =3B>=3BWe have race conditions=3D2C like this=3D2C m3core/src/Text.m3: >>> >=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BVAR fromC= > >> harCache :=3D3D ARRAY CHAR OF T {NIL=3D2C ..}=3D3B (* 1-char texts *)
&g= > >> t=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE Fr= > >> omChar (c: CHAR): T =3D3D
>=3B >=3B>=3B VAR buf: ARRAY [0..0] OF = > >> CHAR=3D3B
>=3B >=3B>=3B BEGIN
>=3B >=3B>=3B IF fromCh= > >> arCache [c] =3D3D NIL THEN
>=3B >=3B>=3B buf [0] :=3D3D c=3D3= > >> B
>=3B >=3B>=3B fromCharCache[c] :=3D3D Text8.New (buf)=3D3B<= > >> br>>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B RETURN fromCha= > >> rCache [c]
>=3B >=3B>=3B END FromChar=3D3B
>=3B >=3B>=3B= > >>
>=3B >=3B>=3B
>=3B >=3B>=3BIt should be:
>=3B >= > >> =3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3BPROCEDURE FromChar (= > >> c: CHAR): T =3D3D
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B VA= > >> R buf: ARRAY [0..0] OF CHAR=3D3B
>=3B >=3B>=3B=3D0A=3D
>=3B &= > >> gt=3B>=3B BEGIN
>=3B >=3B>=3B=3D0A=3D
>=3B >=3B>=3B = > >> IF fromCharCache [c] =3D3D NIL THEN
>=3B >=3B>=3B=3D0A=3D
>= > >> =3B >=3B>=3B buf [0] :=3D3D c=3D3B
>=3B >=3B>=3B=3D0A=3D<= > >> br>>=3B >=3B>=3B WITH a =3D3D Text8.New (buf) DO
>=3B >= > >> =3B>=3B MemoryBarrier()=3D3B
>=3B >=3B>=3B fromCha= > >> rCache[c] :=3D3D a=3D3B
>=3B >=3B>=3B END=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END=3D3B
>=3B >=3B>=3B= > >> =3D0A=3D
>=3B >=3B>=3B RETURN fromCharCache [c]
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B END FromChar=3D3B
>=3B >= > >> =3B>=3B=3D0A=3D
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B &= > >> gt=3B>=3Bto ensure that all of Text8.New() finishes before the assignment= > >> to fromCha=3D
>=3B >=3B>=3BrCache[c] is made.
>=3B >=3B>= > >> =3B
>=3B >=3B>=3B
>=3B >=3B>=3BCan the compiler somehow c= > >> atch these?
>=3B >=3B>=3BI fear they are a small epidemic.
>= > >> =3B >=3B>=3BFor a long time people didn't realize where all the compile= > >> r and processor =3D
>=3B >=3B>=3Bcould reorder.
>=3B >=3B&g= > >> t=3B
>=3B >=3B>=3B
>=3B >=3B>=3BDo we have the right cons= > >> tructs by now to fix them?
>=3B >=3B>=3BI think we do.
>=3B &= > >> gt=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B - Jay
>=3B &= > >> gt=3B>=3B =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_= > >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3BContent-Type: t= > >> ext/html=3B charset=3D"iso-8859-1"
>=3B >=3B>=3BContent-Transfer-E= > >> ncoding: quoted-printable
>=3B >=3B>=3B
>=3B >=3B>=3B<= > >> =3Bhtml>=3B
>=3B >=3B>=3B<=3Bhead>=3B
>=3B >=3B>=3B= > >> <=3Bstyle>=3B<=3B!--
>=3B >=3B>=3B.hmmessage P
>=3B >= > >> =3B>=3B{
>=3B >=3B>=3Bmargin:0px=3D3B
>=3B >=3B>=3Bpadd= > >> ing:0px
>=3B >=3B>=3B}
>=3B >=3B>=3Bbody.hmmessage
>= > >> =3B >=3B>=3B{
>=3B >=3B>=3Bfont-size: 12pt=3D3B
>=3B >= > >> =3B>=3Bfont-family:Calibri
>=3B >=3B>=3B}
>=3B >=3B>=3B= > >> -->=3B<=3B/style>=3B<=3B/head>=3B
>=3B >=3B>=3B<=3Bbod= > >> y class=3D3D'hmmessage'>=3B<=3Bdiv dir=3D3D'ltr'>=3BWe have race cond= > >> itions=3D2C like =3D
>=3B >=3B>=3Bthis=3D2C m3core/src/Text.m3:<= > >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BVAR fromCharCache :=3D3D ARRAY CHAR = > >> OF=3D
>=3B >=3B>=3B T {NIL=3D2C ..}=3D3B (* 1-char texts *)<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar (c: CHAR=3D
>=3B= > >> >=3B>=3B): T =3D3D<=3Bbr>=3B&=3Bnbsp=3D3B VAR buf: ARRAY [0..0]= > >> OF CHAR=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D
>= > >> =3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCha= > >> rCache [c] =3D3D NIL THEN<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D
&g= > >> t=3B >=3B>=3B=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf = > >> [0] :=3D3D c=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B&=3Bn=3D
>=3B >=3B>=3Bbsp=3D3B&=3Bnbsp=3D3B fromCharCac= > >> he[c] :=3D3D Text8.New (buf)=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp= > >> =3D3B=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RETURN fromCharCache [c]<=3B= > >> br>=3B&=3Bnb=3D
>=3B >=3B>=3Bsp=3D3B END FromChar=3D3B<=3Bb= > >> r>=3B<=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3BPROCEDURE FromCha=3D
>=3B >=3B>=3Br (c: CHAR): T = > >> =3D3D<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B VAR buf:= > >> ARRAY [0..0] OF CHAR=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B BEGIN<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp= > >> =3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B IF fromCharCache [c] =3D3D NIL THEN&l= > >> t=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B= > >> &=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B buf [0] :=3D3D c=3D3B<= > >> =3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B WITH a =3D3D Text8.New (buf= > >> ) DO<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3Bnbsp=3D3B&=3Bnbsp=3D3= > >> B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbs= > >> p=3D3B MemoryBarrier()=3D3B<=3B=3D
>=3B >=3B>=3Bbr>=3B&=3Bn= > >> bsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&am= > >> p=3Bnbsp=3D3B&=3Bnbsp=3D3B fromCharCache[c=3D
>=3B >=3B>=3B] := > >> =3D3D a=3D3B<=3Bbr>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&= > >> amp=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&= > >> =3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B END=3D3B<=3Bbr>=3B=3D0A=3D= > >>
>=3B >=3B>=3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B&=3Bnbsp=3D3B RET= > >> URN fromCharCache [c]<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B&=3Bnb= > >> sp=3D3B END FromChar=3D3B<=3Bbr>=3B=3D0A=3D
>=3B >=3B>=3B<= > >> =3Bbr>=3B<=3Bbr>=3Bto ensure that all of Text8.New() finishes before = > >> the assignment to=3D
>=3B >=3B>=3B fromCharCache[c] is made.<=3B= > >> br>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compiler somehow catch these?= > >> =3D
>=3B >=3B>=3B<=3Bbr>=3BI fear they are a small epidemic.&l= > >> t=3Bbr>=3BFor a long time people didn't real=3D
>=3B >=3B>=3Bize= > >> where all the compiler and processor could reorder.<=3Bbr>=3B<=3Bbr&= > >> gt=3B<=3Bbr>=3BDo we ha=3D
>=3B >=3B>=3Bve the right construct= > >> s by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr>=3B<=3Bbr>= > >> =3B<=3Bbr>=3B -=3D
>=3B >=3B>=3B Jay<=3Bbr>=3B = > >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B<=3B/html>=3B= > >> =3D
>=3B >=3B>=3B
>=3B >=3B>=3B--_5f23896e-68ee-44da-82dd= > >> -311cd8c58979_--
> >> = > >> > >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Aug 27 20:34:51 2012 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 27 Aug 2012 11:34:51 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> Message-ID: <20120827183451.976971A2094@async.async.caltech.edu> yeah I agree. processor 1 performs... x := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent. If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive. In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2. (Or Exclusive, if P1 writes it back and invalidates.) This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global. The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice. I'd suspect that a multiprocessor 21264 could be. But I do agree it's a bug relative to the Alpha Architecture Manual. A very nasty one at that. Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily. The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way. It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory. I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature? How are t= >he >> > pthreads doing? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> > Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running. How is that possible? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere. Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run. Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though. But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>> Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory. But is it important? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>> Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>> END=3D3D3B >> >>>>> RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>> buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> WITH a =3D3D3D Text8.New (buf) DO >> >>>>> MemoryBarrier()=3D3D3B >> >>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running. How is that possible? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere. = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run. Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >t=3D3B IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >B RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B =3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B =3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily. The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way. It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere. Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory. But is it important? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= > buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = > <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B =3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From dabenavidesd at yahoo.es Mon Aug 27 23:54:16 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 27 Aug 2012 22:54:16 +0100 (BST) Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <1346104456.75314.YahooMailClassic@web29706.mail.ird.yahoo.com> Hi all: I believe there shouldn't be so many for not being counted with fingers of one hand. The memory model was specified and model checked with TLA by Lamport and publicly available for bug finders contest. However being an UNSAFE MODULE I suspect this was intended to explode that relaxed restriction. Also probable a C weak semantics could make things worse or better than they are, so ... Thanks in advance --- El lun, 27/8/12, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] race conditions in lock-free code... Para: "Jay K" CC: m3devel at elegosoft.com Fecha: lunes, 27 de agosto, 2012 13:34 yeah I agree. processor 1 performs... x? := alloc_new_text x^ := character return x global := x processor 2 performs IF global # NIL THEN use(global) END now if out of all this, x^ := character is held back, e.g., in a processor write buffer, and if use(global) involves performing global^, the value of global^ will be the value of global^ just after x := alloc_new_text and before x^ := character. I believe very aggressive architectures such as Alpha permit reordering to that extent.? If memory serves, on Alpha, if you do not perform a Memory Barrier, pretty much nothing can be inferred on one processor about the ordering of operations on another. X86 I am not sure is that aggressive.? In a hardware-synchronized multiprocessor environment without barrier operations using e.g. the Illinois/MESI protocol (doesn't X86?), at the point x^ := character is performed, the cache coherence protocol has to guarantee the cache line is in state Exclusive, which means it has to be evicted from the cache in processor 2 and when use(global) comes around the protocol will have to ensure it is in Shared state on processor 2.? (Or Exclusive, if P1 writes it back and invalidates.)? This would guarantee that you could not actually see a very old value for global^ at the same time as a newer value for global.? The protocol guarantees that the relative ordering of writes is maintained across processors. Of course I don't know whether Alphas are actually that aggressive in practice.? I'd suspect that a multiprocessor 21264 could be.? But I do agree it's a bug relative to the Alpha Architecture Manual.? A very nasty one at that. ? ? Mika Jay K writes: >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >It is NOT benign. >The global can be written before the initialization of what it points to is= > done. >The result is wrong. > > >It IS benign if multiple threads go and allocate separate multiple initiali= >zed TEXTs. >That's not the problem. > > > - Jay > > >> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >> From: hosking at cs.purdue.edu >> Subject: Re: [M3devel] race conditions in lock-free code... >> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >> To: mika at async.caltech.edu >>=20 >> So why is this anything other than benign? It's a cache and there's no pr= >oblem if different threads get different texts. Don't cripple performance u= >nnecessarily.? The collector will reclaim any unreachable text. >>=20 >> Sent from my iPad >>=20 >> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >wrote: >>=20 >> > Oh I see now. >> >=20 >> > It's not that the processor reorders in any tricky way.? It is that the >> > write performed by Text8.New could be sitting around processor 1's cach= >e >> > indefinitely without actually getting written back to main memory. >> > Processor 2 might see the new pointer=2C but when reading the memory po= >inted >> > to=2C would get uninitialized memory.? I think this can happen on Alpha= > but >> > maybe not X86 (without further reorderings anyhow)? >> >=20 >> > Any chance any of the pthreads bugs could be of this nature?? How are t= >he >> > pthreads doing?? I gathered some bugs were fixed but I haven't tried th= >e >> > system for a while. >> >=20 >> >? ???Mika >> >=20 >> >=20 >> > Jay K writes: >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> The processor can reorder. Across call/ret at least in general. >> >>=20 >> >>=20 >> >> The compiler can also reorder=3D2C if you have whole-program-optimizat= >ion=3D2C =3D >> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= > "LTO")=3D >> >> . (We don't=3D2C but real world systems certainly do.) >> >>=20 >> >>=20 >> >> - Jay >> >>=20 >> >>=20 >> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >caltech.=3D >> >> edu >> >>> Subject: Re: [M3devel] race conditions in lock-free code... >> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >> >>> From: mika at async.caltech.edu >> >>> =3D20 >> >>> Sorry=3D2C I can't seem to quote your message. >> >>> =3D20 >> >>> You say that Text8.New's result can be assigned to fromCharCache[c] >> >>> before Text8.New is done running.? How is that possible?? Surely >> >>> Text8.New has to calculate its result before it can be assigned >> >>> anywhere.? Yes possibly FromChar allocates space for the result of >> >>> Text8.New but the assignment of whatever appears in that space can't >> >>> happen until *after* Text8.New has run.? Unless you are saying that >> >>> the running of Text8.New and FromChar are intertwined somehow..? >> >>> I know compilers do some odd things but that doesn't sound right... >> >>> =3D20 >> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >> >>> right=3D2C though.? But it doesn't... >> >>> =3D20 >> >>> Am I missing something? >> >>> =3D20 >> >>>? ???Mika >> >>> =3D20 >> >>> Mika Nystrom writes: >> >>>>=20 >> >>>> Yeah it's a race condition=3D2C in theory.? But is it important?? If= > you g=3D >> >> et >> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR.? = >But no >> >>>> one ever said FromChar was guaranteed to return the same pointer whe= >n >> >>>> you call it with the same CHAR... >> >>>>=20 >> >>>>? ? Mika >> >>>>=20 >> >>>> Jay K writes: >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >> >>>>>=20 >> >>>>>=20 >> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >* 1-char t=3D >> >> exts *) >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> BEGIN >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>>? ???fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >> >>>>>???END=3D3D3B >> >>>>>???RETURN fromCharCache [c] >> >>>>> END FromChar=3D3D3B >> >>>>>=20 >> >>>>>=20 >> >>>>> It should be: >> >>>>>=20 >> >>>>>=20 >> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >> >>>>> =3D3D0A=3D3D >> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>> BEGIN >> >>>>> =3D3D0A=3D3D >> >>>>>???IF fromCharCache [c] =3D3D3D NIL THEN >> >>>>> =3D3D0A=3D3D >> >>>>>? ???buf [0] :=3D3D3D c=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>? ???WITH a =3D3D3D Text8.New (buf) DO >> >>>>>? ? ???MemoryBarrier()=3D3D3B >> >>>>>? ? ???fromCharCache[c] :=3D3D3D a=3D3D3B >> >>>>>? ? END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???END=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>???RETURN fromCharCache [c] >> >>>>> =3D3D0A=3D3D >> >>>>> END FromChar=3D3D3B >> >>>>> =3D3D0A=3D3D >> >>>>>=20 >> >>>>>=20 >> >>>>> to ensure that all of Text8.New() finishes before the assignment to= > fro=3D >> >> mCha=3D3D >> >>>>> rCache[c] is made. >> >>>>>=20 >> >>>>>=20 >> >>>>> Can the compiler somehow catch these? >> >>>>> I fear they are a small epidemic. >> >>>>> For a long time people didn't realize where all the compiler and pr= >oces=3D >> >> sor =3D3D >> >>>>> could reorder. >> >>>>>=20 >> >>>>>=20 >> >>>>> Do we have the right constructs by now to fix them? >> >>>>> I think we do. >> >>>>>=20 >> >>>>>=20 >> >>>>> - Jay >> >>>>>? ? ? ? ? ? ? ? ? ? ? ???=3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >> >>>>> Content-Transfer-Encoding: quoted-printable >> >>>>>=20 >> >>>>> >> >>>>> >> >>>>> >> >>>>>
We have race con= >ditions=3D >> >> =3D3D2C like =3D3D >> >>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >3D ARRAY=3D >> >> CHAR OF=3D3D >> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >omChar =3D >> >> (c: CHAR=3D3D >> >>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3Br> =3D3D=3D >> >> 3B BEGIN
=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
&=3D >> >> nbsp=3D3D3B =3D3D >> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >3B
 =3D3D3B=3D >> >>  =3D3D3B =3D3D3B&n=3D3D >> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >3D3B
&nbs=3D >> >> p=3D3D3B =3D3D3B=3D3D >> >>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >URN fromCharCa=3D >> >> che [c]
&nb=3D3D >> >>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >ROCEDURE=3D >> >> FromCha=3D3D >> >>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >IL THEN
=3D >> >> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >] :=3D3D3D c=3D3D3=3D >> >> B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= > =3D3D3D Text8.N=3D >> >> ew (buf) DO
&=3D3D >> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B =3D3D3B M=3D >> >> emoryBarrier()=3D3D3B<=3D3D >> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >p=3D3D3B =3D3D=3D >> >> 3B fromCharCache[c=3D3D >> >>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >3D3B END=3D3D3B> >>> =3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >=3D3D0A=3D3D >> >>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >> >>>>>

to ensure that all of Text8.New() finishes before the assig= >nmen=3D >> >> t to=3D3D >> >>>>> fromCharCache[c] is made.


Can the compiler somehow catch= > th=3D >> >> ese?=3D3D >> >>>>>
I fear they are a small epidemic.
For a long time people did= >n't =3D >> >> real=3D3D >> >>>>> ize where all the compiler and processor could reorder.


= >Do w=3D >> >> e ha=3D3D >> >>>>> ve the right constructs by now to fix them?
I think we do.
r>> >> r> -=3D3D >> >>>>> Jay
? ? ? ? ? ? ? ? ? ? ? ???
>> >>>>> =3D3D >> >>>>>=20 >> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >> >>? ? ? ? ? ? ? ? ? ? ? ???=3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >> >> Content-Type: text/html=3B charset=3D"iso-8859-1" >> >> Content-Transfer-Encoding: quoted-printable >> >>=20 >> >> >> >> >> >> >> >>
The processor can reord= >er. Acros=3D >> >> s call/ret at least in general.


The compiler can also reord= >er=3D2C=3D >> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >oft "L=3D >> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= > world sy=3D >> >> stems certainly do.)


 =3D3B- Jay


d=3D3D"Sk=3D >> >> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >el at elegos=3D >> >> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= > race con=3D >> >> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >:32:48 -0=3D >> >> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >Sorry=3D2C I =3D >> >> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >t Text8.N=3D >> >> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >t8.New =3D >> >> is done running.? How is that possible?? Surely
>=3D3B Text8.New = >has to =3D >> >> calculate its result before it can be assigned
>=3D3B anywhere.? = >Yes pos=3D >> >> sibly FromChar allocates space for the result of
>=3D3B Text8.New= > but th=3D >> >> e assignment of whatever appears in that space can't
>=3D3B happe= >n until=3D >> >> *after* Text8.New has run.? Unless you are saying that
>=3D3B the= > runni=3D >> >> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >know co=3D >> >> mpilers do some odd things but that doesn't sound right...
>=3D3B= >
&g=3D >> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= > be
=3D >> >> >=3D3B right=3D2C though.? But it doesn't...
>=3D3B
>=3D3= >B Am I missin=3D >> >> g something?
>=3D3B
>=3D3B? ? ? Mika
>=3D3B
>=3D= >3B Mika Nystr=3D >> >> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >ondition=3D2C i=3D >> >> n theory.? But is it important?? If you get
>=3D3B >=3D3Bthe wr= >ong copy=3D >> >> =3D2C there will be two TEXTs representing one CHAR.? But no
>=3D= >3B >=3D3B=3D >> >> one ever said FromChar was guaranteed to return the same pointer when<= >br>&g=3D >> >> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >br>>=3D3B >=3D >> >> =3D3B? ???Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >=3D3B>=3D3BCon=3D >> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >3B>=3D3BConte=3D >> >> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>=3D3B >=3D >> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >c/Text.m3:> >>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >t=3D3B>=3D3BVAR fromC=3D >> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >ts *)
&g=3D >> >> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3BPROCEDURE Fr=3D >> >> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B? VAR buf: ARR= >AY [0..0] OF =3D >> >> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B&g= >t=3D3B? ? IF fromCh=3D >> >> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B? ? ? buf [0]= > :=3D3D3D c=3D3D3=3D >> >> B
>=3D3B >=3D3B>=3D3B? ? ? fromCharCache[c] :=3D3D3D Text8.Ne= >w (buf)=3D3D3B<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? END=3D3D3B
>=3D3B >=3D3B>=3D3= >B? ? RETURN fromCha=3D >> >> rCache [c]
>=3D3B >=3D3B>=3D3B? END FromChar=3D3D3B
>=3D= >3B >=3D3B>=3D3B=3D >> >>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >:
>=3D3B >=3D >> >> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >BPROCEDURE FromChar (=3D >> >> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B? VA=3D >> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >=3D3D
>=3D3B &=3D >> >> gt=3D3B>=3D3B? BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >=3D3B >=3D3B>=3D3B???=3D >> >> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >D0A=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >=3D3B>=3D3B=3D3D0A=3D3D<=3D >> >> br>>=3D3B >=3D3B>=3D3B? ? ? WITH a =3D3D3D Text8.New (buf) DO>>=3D3B >=3D >> >> =3D3B>=3D3B? ? ? ? MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >3B? ? ? ? fromCha=3D >> >> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B? ???END=3D3D3= >B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? END=3D3D3Br>>=3D3B >=3D3B>=3D3B=3D >> >> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? ? RETURN fromCharCache [c]<= >br>>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B? END FromChar= >=3D3D3B
>=3D3B >=3D >> >> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B
>=3D3B &=3D >> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >ssignment=3D >> >> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >=3D3B >=3D3B>=3D >> >> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= > compiler somehow c=3D >> >> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >ic.
>=3D >> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >the compile=3D >> >> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >t=3D3B >=3D3B&g=3D >> >> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >have the right cons=3D >> >> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >br>>=3D3B &=3D >> >> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >=3D3B - Jay
>=3D3B &=3D >> >> gt=3D3B>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
>=3D3B >=3D3B>= >=3D3B
>=3D3B >=3D3B>=3D3B--_=3D >> >> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >ent-Type: t=3D >> >> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >tent-Transfer-E=3D >> >> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D >> >> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>=3D3B >=3D3B>=3D3B=3D >> >> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >e P
>=3D3B >=3D >> >> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >3B >=3D3B>=3D3Bpadd=3D >> >> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >.hmmessage
>=3D >> >> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >3D3B
>=3D3B >=3D >> >> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >=3D3B >=3D3B>=3D3B=3D >> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >=3D3B>=3D3B<=3D3Bbod=3D >> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= > have race cond=3D >> >> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >e/src/Text.m3:<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >=3D3D3D ARRAY CHAR =3D >> >> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >AR=3D3D
>=3D3B=3D >> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= > buf: ARRAY [0..0]=3D >> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >t=3D3B=3D3D
>=3D >> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >Bnbsp=3D3D3B IF fromCha=3D >> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >=3D3Bnbsp=3D3D
&g=3D >> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B buf =3D >> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >sp=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >nbsp=3D3D3B fromCharCac=3D >> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D >> >> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >3B<=3D3Bbr>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >arCache [c]<=3D3B=3D >> >> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >romChar=3D3D3B<=3D3Bb=3D >> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >t=3D3B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >=3D3Br (c: CHAR): T =3D >> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >=3D3Bnbsp=3D3D3B VAR buf:=3D >> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >=3D3B>=3D3B&=3D3Bnbsp=3D >> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >=3D3D3D NIL THEN&l=3D >> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B=3D >> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >=3D3D3D c=3D3D3B<=3D >> >> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >3D3D Text8.New (buf=3D >> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >=3D3D3B&=3D3Bnbsp=3D3D3=3D >> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >nbsp=3D3D3B&=3D3Bnbs=3D >> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >=3D3Bbr>=3D3B&=3D3Bn=3D >> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >&=3D3Bnbsp=3D3D3B&am=3D >> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= > >=3D3B>=3D3B] :=3D >> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >3D3B&=3D3Bnbsp=3D3D3B&=3D >> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >3B >=3D3B>=3D3B&=3D >> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >=3D3Bbr>=3D3B=3D3D0A=3D3D=3D >> >>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >amp=3D3Bnbsp=3D3D3B RET=3D >> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >3B>=3D3B&=3D3Bnb=3D >> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >B >=3D3B>=3D3B<=3D >> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >ishes before =3D >> >> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >s made.<=3D3B=3D >> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= > catch these?=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >small epidemic.&l=3D >> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >t=3D3B>=3D3Bize=3D >> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >lt=3D3Bbr&=3D >> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >the right construct=3D >> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >B<=3D3Bbr>=3D >> >> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >Bbr>=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D >> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= ><=3D3B/html>=3D3B=3D >> >> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >896e-68ee-44da-82dd=3D >> >> -311cd8c58979_--
? ? ? ? ? ? ? ? ? ? ? ???
>> >> =3D >> >>=20 >> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- > ??? ???????? ?????? ??? ? = > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
It is NOT benign.
The global = >can be written before the initialization of what it points to is done.
T= >he result is wrong.


It IS benign if multiple threads go and allo= >cate separate multiple initialized TEXTs.
That's not the problem.
>
 =3B- Jay


&g= >t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >-free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >r than benign? It's a cache and there's no problem if different threads get= > different texts. Don't cripple performance unnecessarily.? The collector w= >ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= > any tricky way.? It is that the
>=3B >=3B write performed by Text8.= >New could be sitting around processor 1's cache
>=3B >=3B indefinite= >ly without actually getting written back to main memory.
>=3B >=3B P= >rocessor 2 might see the new pointer=2C but when reading the memory pointed= >
>=3B >=3B to=2C would get uninitialized memory.? I think this can h= >appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >ads bugs could be of this nature?? How are the
>=3B >=3B pthreads do= >ing?? I gathered some bugs were fixed but I haven't tried the
>=3B >= >=3B system for a while.
>=3B >=3B
>=3B >=3B? ???Mika
>= >=3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= > >=3B>=3B The processor can reorder. Across call/ret at least in genera= >l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= > but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = > How is that possible?? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >o calculate its result before it can be assigned
>=3B >=3B>=3B>= >=3B anywhere.? Yes possibly FromChar allocates space for the result of
&= >gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >New has run.? Unless you are saying that
>=3B >=3B>=3B>=3B the r= >unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >=3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= > Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>=3B>=3B>=3B right=3D2C though.? But it doesn't...
>=3B >=3B&g= >t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B? ???Mika
>= >=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >Yeah it's a race condition=3D2C in theory.? But is it important?? If you g= >=3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >copy=3D2C there will be two TEXTs representing one CHAR.? But no
>=3B = >>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= > same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >=3B>=3B? ? Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= > exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGINr>>=3B >=3B>=3B>=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL = >THEN
>=3B >=3B>=3B>=3B>=3B>=3B? ???buf [0] :=3D3D3D c=3D3D3B= >
>=3B >=3B>=3B>=3B>=3B>=3B? ???fromCharCache[c] :=3D3D3D Tex= >t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B???END=3D3D3Br>>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c]
>= >=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B???IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >? ???buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B? ???WITH a =3D3D3D Text8= >.New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???MemoryBarrier(= >)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? ???fromCharCache[c] := >=3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B? ? END=3D3D3B
= >>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >t=3B>=3B>=3B???END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B???RETURN fromCharCache [c= >]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= > ensure that all of Text8.New() finishes before the assignment to fro=3D>>=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= > long time people didn't realize where all the compiler and proces=3D
&g= >t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >=3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >
>=3B >=3B>=3B>=3B>=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= > >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >--
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >=3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >.hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= ><=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >=3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= > >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= ><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >=3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >to ensure that all of Text8.New() finishes before the assignmen=3D
>= >=3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B? ? ? ? ? ? ? ? ? ? ???= >? <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >=3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >--
>=3B >=3B>=3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>= >=3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >=3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= > >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= ><=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >"). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >=3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >: Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >ning.? How is that possible?? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >igned<=3Bbr>=3B&=3Bgt=3D3B anywhere.? Yes pos=3D
>=3B >=3B>= >=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >=3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >gt=3B >=3B>=3B *after* Text8.New has run.? Unless you are saying that&l= >t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >ght=3D2C though.? But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >=3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B? ? ? Mika<=3Bbr>=3B&= >amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >t=3B>=3B n theory.? But is it important?? If you get<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >will be two TEXTs representing one CHAR.? But no<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >br>>=3B >=3B>=3B =3D3B? ???Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= > charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= > race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >=3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B? VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? BEGIN&l= >t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? IF fromCh=3D
= >>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ???= > fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCha=3D= >
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? VA=3D
>=3B >=3B>=3B R bu= >f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >gt=3B>=3B gt=3D3B&=3Bgt=3D3B? BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B&=3Bgt=3D3B???=3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B? ? ? buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? WITH a =3D3D3D Text8.= >New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >=3B =3D3B&=3Bgt=3D3B? ? ? ? MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? ? ? fromCha=3D
>=3B >=3B>= >=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >amp=3Bgt=3D3B? ???END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B? ? END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B? ? RETURN fromCharCache [c= >]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B? END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >=3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >=3B>=3B gt=3D3B&=3Bgt=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D3D<=3Bbr>=3B= >&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= > >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >&=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >=3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >=3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >=3D3B? ? ? ? ? ? ? ? ? ? ? ???=3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= > >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >v>=3B? ? ? ? ? ? ? ? ? ? ? ???<=3B/div>=3B<=3B/body>=3B
>=3B= > >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
??? ???????? ?????? ??? ? v> >= > >--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Aug 28 02:40:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 20:40:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <20120827183451.976971A2094@async.async.caltech.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu>, <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> Message-ID: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > > yeah I agree. > > processor 1 performs... > > x := alloc_new_text > x^ := character > return x > global := x > > processor 2 performs > > IF global # NIL THEN use(global) END > > now if out of all this, x^ := character is held back, e.g., in a processor > write buffer, and if use(global) involves performing global^, the value > of global^ will be the value of global^ just after x := alloc_new_text > and before x^ := character. > > I believe very aggressive architectures such as Alpha permit reordering > to that extent. If memory serves, on Alpha, if you do not perform a > Memory Barrier, pretty much nothing can be inferred on one processor > about the ordering of operations on another. > > X86 I am not sure is that aggressive. In a hardware-synchronized > multiprocessor environment without barrier operations using e.g. the > Illinois/MESI protocol (doesn't X86?), at the point x^ := character is > performed, the cache coherence protocol has to guarantee the cache line > is in state Exclusive, which means it has to be evicted from the cache > in processor 2 and when use(global) comes around the protocol will > have to ensure it is in Shared state on processor 2. (Or Exclusive, > if P1 writes it back and invalidates.) This would guarantee that you > could not actually see a very old value for global^ at the same time > as a newer value for global. The protocol guarantees that the relative > ordering of writes is maintained across processors. > > Of course I don't know whether Alphas are actually that aggressive in > practice. I'd suspect that a multiprocessor 21264 could be. But I do > agree it's a bug relative to the Alpha Architecture Manual. A very nasty > one at that. > > Mika > > > Jay K writes: >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> It is NOT benign. >> The global can be written before the initialization of what it points to is= >> done. >> The result is wrong. >> >> >> It IS benign if multiple threads go and allocate separate multiple initiali= >> zed TEXTs. >> That's not the problem. >> >> >> - Jay >> >> >>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>> From: hosking at cs.purdue.edu >>> Subject: Re: [M3devel] race conditions in lock-free code... >>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>> To: mika at async.caltech.edu >>> =20 >>> So why is this anything other than benign? It's a cache and there's no pr= >> oblem if different threads get different texts. Don't cripple performance u= >> nnecessarily. The collector will reclaim any unreachable text. >>> =20 >>> Sent from my iPad >>> =20 >>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >> wrote: >>> =20 >>>> Oh I see now. >>>> =20 >>>> It's not that the processor reorders in any tricky way. It is that the >>>> write performed by Text8.New could be sitting around processor 1's cach= >> e >>>> indefinitely without actually getting written back to main memory. >>>> Processor 2 might see the new pointer=2C but when reading the memory po= >> inted >>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >> but >>>> maybe not X86 (without further reorderings anyhow)? >>>> =20 >>>> Any chance any of the pthreads bugs could be of this nature? How are t= >> he >>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >> e >>>> system for a while. >>>> =20 >>>> Mika >>>> =20 >>>> =20 >>>> Jay K writes: >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> The processor can reorder. Across call/ret at least in general. >>>>> =20 >>>>> =20 >>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >> ion=3D2C =3D >>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >> "LTO")=3D >>>>> . (We don't=3D2C but real world systems certainly do.) >>>>> =20 >>>>> =20 >>>>> - Jay >>>>> =20 >>>>> =20 >>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >> caltech.=3D >>>>> edu >>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> =3D20 >>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>> =3D20 >>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>> before Text8.New is done running. How is that possible? Surely >>>>>> Text8.New has to calculate its result before it can be assigned >>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>> =3D20 >>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>> right=3D2C though. But it doesn't... >>>>>> =3D20 >>>>>> Am I missing something? >>>>>> =3D20 >>>>>> Mika >>>>>> =3D20 >>>>>> Mika Nystrom writes: >>>>>>> =20 >>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >> you g=3D >>>>> et >>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >> But no >>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >> n >>>>>>> you call it with the same CHAR... >>>>>>> =20 >>>>>>> Mika >>>>>>> =20 >>>>>>> Jay K writes: >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >> * 1-char t=3D >>>>> exts *) >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> BEGIN >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> RETURN fromCharCache [c] >>>>>>>> END FromChar=3D3D3B >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> It should be: >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>> =3D3D0A=3D3D >>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> BEGIN >>>>>>>> =3D3D0A=3D3D >>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>> =3D3D0A=3D3D >>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> END=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> RETURN fromCharCache [c] >>>>>>>> =3D3D0A=3D3D >>>>>>>> END FromChar=3D3D3B >>>>>>>> =3D3D0A=3D3D >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >> fro=3D >>>>> mCha=3D3D >>>>>>>> rCache[c] is made. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Can the compiler somehow catch these? >>>>>>>> I fear they are a small epidemic. >>>>>>>> For a long time people didn't realize where all the compiler and pr= >> oces=3D >>>>> sor =3D3D >>>>>>>> could reorder. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>> I think we do. >>>>>>>> =20 >>>>>>>> =20 >>>>>>>> - Jay >>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>> =20 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>
We have race con= >> ditions=3D >>>>> =3D3D2C like =3D3D >>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >> 3D ARRAY=3D >>>>> CHAR OF=3D3D >>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >> omChar =3D >>>>> (c: CHAR=3D3D >>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B> r> =3D3D=3D >>>>> 3B BEGIN
=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
&=3D >>>>> nbsp=3D3D3B =3D3D >>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >> 3B
 =3D3D3B=3D >>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >> 3D3B
&nbs=3D >>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >> URN fromCharCa=3D >>>>> che [c]
&nb=3D3D >>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >> ROCEDURE=3D >>>>> FromCha=3D3D >>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >> IL THEN
=3D >>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >> ] :=3D3D3D c=3D3D3=3D >>>>> B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >> =3D3D3D Text8.N=3D >>>>> ew (buf) DO
&=3D3D >>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B =3D3D3B M=3D >>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >> p=3D3D3B =3D3D=3D >>>>> 3B fromCharCache[c=3D3D >>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >> 3D3B END=3D3D3B>>>>> =3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >> =3D3D0A=3D3D >>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >> nmen=3D >>>>> t to=3D3D >>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >> th=3D >>>>> ese?=3D3D >>>>>>>>
I fear they are a small epidemic.
For a long time people did= >> n't =3D >>>>> real=3D3D >>>>>>>> ize where all the compiler and processor could reorder.


= >> Do w=3D >>>>> e ha=3D3D >>>>>>>> ve the right constructs by now to fix them?
I think we do.
> r>>>>> r> -=3D3D >>>>>>>> Jay
>>>>>>>> =3D3D >>>>>>>> =20 >>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> =20 >>>>> >>>>> >>>>> >>>>>
The processor can reord= >> er. Acros=3D >>>>> s call/ret at least in general.


The compiler can also reord= >> er=3D2C=3D >>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >> oft "L=3D >>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >> world sy=3D >>>>> stems certainly do.)


 =3D3B- Jay


> d=3D3D"Sk=3D >>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >> el at elegos=3D >>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >> race con=3D >>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >> :32:48 -0=3D >>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >> Sorry=3D2C I =3D >>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >> t Text8.N=3D >>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >> t8.New =3D >>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >> has to =3D >>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >> Yes pos=3D >>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >> but th=3D >>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >> n until=3D >>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >> runni=3D >>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >> know co=3D >>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>
&g=3D >>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >> be
=3D >>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >> B Am I missin=3D >>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >> 3B Mika Nystr=3D >>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >> ondition=3D2C i=3D >>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >> ong copy=3D >>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >> 3B >=3D3B=3D >>>>> one ever said FromChar was guaranteed to return the same pointer when<= >> br>&g=3D >>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >> br>>=3D3B >=3D >>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >> =3D3B>=3D3BCon=3D >>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >> 3B>=3D3BConte=3D >>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>> >=3D3B >=3D >>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >> c/Text.m3:>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >> t=3D3B>=3D3BVAR fromC=3D >>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >> ts *)
&g=3D >>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3BPROCEDURE Fr=3D >>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >> AY [0..0] OF =3D >>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >> t=3D3B IF fromCh=3D >>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >> :=3D3D3D c=3D3D3=3D >>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >> w (buf)=3D3D3B<=3D >>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >> B RETURN fromCha=3D >>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >> 3B >=3D3B>=3D3B=3D >>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >> :
>=3D3B >=3D >>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >> BPROCEDURE FromChar (=3D >>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B VA=3D >>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >> =3D3D
>=3D3B &=3D >>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >> =3D3B >=3D3B>=3D3B =3D >>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >> D0A=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>> >=3D3B >=3D >>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >> 3B fromCha=3D >>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >> B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B> r>>=3D3B >=3D3B>=3D3B=3D >>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >> br>>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >> =3D3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B
>=3D3B &=3D >>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >> ssignment=3D >>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >> =3D3B >=3D3B>=3D >>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >> compiler somehow c=3D >>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >> ic.
>=3D >>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >> the compile=3D >>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >> t=3D3B >=3D3B&g=3D >>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >> have the right cons=3D >>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >> br>>=3D3B &=3D >>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >> =3D3B - Jay
>=3D3B &=3D >>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >> ent-Type: t=3D >>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >> tent-Transfer-E=3D >>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D >>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >> >=3D3B >=3D3B>=3D3B=3D >>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >> e P
>=3D3B >=3D >>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >> 3B >=3D3B>=3D3Bpadd=3D >>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >> .hmmessage
>=3D >>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >> 3D3B
>=3D3B >=3D >>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >> =3D3B >=3D3B>=3D3B=3D >>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >> =3D3B>=3D3B<=3D3Bbod=3D >>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >> have race cond=3D >>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >> e/src/Text.m3:<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >> =3D3D3D ARRAY CHAR =3D >>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >> AR=3D3D
>=3D3B=3D >>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >> buf: ARRAY [0..0]=3D >>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >> t=3D3B=3D3D
>=3D >>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >> Bnbsp=3D3D3B IF fromCha=3D >>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >> =3D3Bnbsp=3D3D
&g=3D >>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B buf =3D >>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >> sp=3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B fromCharCac=3D >>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >> 3B<=3D3Bbr>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >> arCache [c]<=3D3B=3D >>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >> romChar=3D3D3B<=3D3Bb=3D >>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >> t=3D3B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >> =3D3Br (c: CHAR): T =3D >>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >> >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >> =3D3D3D NIL THEN&l=3D >>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >> =3D3D3D c=3D3D3B<=3D >>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >> 3D3D Text8.New (buf=3D >>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >> nbsp=3D3D3B&=3D3Bnbs=3D >>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >> &=3D3Bnbsp=3D3D3B&am=3D >>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >> >=3D3B>=3D3B] :=3D >>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >> 3B >=3D3B>=3D3B&=3D >>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >> amp=3D3Bnbsp=3D3D3B RET=3D >>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >> 3B>=3D3B&=3D3Bnb=3D >>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >> B >=3D3B>=3D3B<=3D >>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >> ishes before =3D >>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >> s made.<=3D3B=3D >>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >> catch these?=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >> small epidemic.&l=3D >>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >> t=3D3B>=3D3Bize=3D >>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >> lt=3D3Bbr&=3D >>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >> the right construct=3D >>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >> B<=3D3Bbr>=3D >>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >> Bbr>=3D3B =3D >>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >> <=3D3B/html>=3D3B=3D >>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >> 896e-68ee-44da-82dd=3D >>>>> -311cd8c58979_--
>>>>> =3D >>>>> =20 >>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
It is NOT benign.
The global = >> can be written before the initialization of what it points to is done.
T= >> he result is wrong.


It IS benign if multiple threads go and allo= >> cate separate multiple initialized TEXTs.
That's not the problem.
>>
 =3B- Jay


&g= >> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >> r than benign? It's a cache and there's no problem if different threads get= >> different texts. Don't cripple performance unnecessarily. The collector w= >> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >> ly without actually getting written back to main memory.
>=3B >=3B P= >> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >> ensure that all of Text8.New() finishes before the assignment to fro=3D>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >> long time people didn't realize where all the compiler and proces=3D
&g= >> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
> v> >> = >> >> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- From jay.krell at cornell.edu Tue Aug 28 04:45:45 2012 From: jay.krell at cornell.edu (Jay) Date: Mon, 27 Aug 2012 19:45:45 -0700 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... - Jay (briefly/pocket-sized-computer-aka-phone) On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. > > Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. > > It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile +1 765 427 5484 > > > > > > On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: > >> >> yeah I agree. >> >> processor 1 performs... >> >> x := alloc_new_text >> x^ := character >> return x >> global := x >> >> processor 2 performs >> >> IF global # NIL THEN use(global) END >> >> now if out of all this, x^ := character is held back, e.g., in a processor >> write buffer, and if use(global) involves performing global^, the value >> of global^ will be the value of global^ just after x := alloc_new_text >> and before x^ := character. >> >> I believe very aggressive architectures such as Alpha permit reordering >> to that extent. If memory serves, on Alpha, if you do not perform a >> Memory Barrier, pretty much nothing can be inferred on one processor >> about the ordering of operations on another. >> >> X86 I am not sure is that aggressive. In a hardware-synchronized >> multiprocessor environment without barrier operations using e.g. the >> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >> performed, the cache coherence protocol has to guarantee the cache line >> is in state Exclusive, which means it has to be evicted from the cache >> in processor 2 and when use(global) comes around the protocol will >> have to ensure it is in Shared state on processor 2. (Or Exclusive, >> if P1 writes it back and invalidates.) This would guarantee that you >> could not actually see a very old value for global^ at the same time >> as a newer value for global. The protocol guarantees that the relative >> ordering of writes is maintained across processors. >> >> Of course I don't know whether Alphas are actually that aggressive in >> practice. I'd suspect that a multiprocessor 21264 could be. But I do >> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >> one at that. >> >> Mika >> >> >> Jay K writes: >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/plain; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> It is NOT benign. >>> The global can be written before the initialization of what it points to is= >>> done. >>> The result is wrong. >>> >>> >>> It IS benign if multiple threads go and allocate separate multiple initiali= >>> zed TEXTs. >>> That's not the problem. >>> >>> >>> - Jay >>> >>> >>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>> From: hosking at cs.purdue.edu >>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>> To: mika at async.caltech.edu >>>> =20 >>>> So why is this anything other than benign? It's a cache and there's no pr= >>> oblem if different threads get different texts. Don't cripple performance u= >>> nnecessarily. The collector will reclaim any unreachable text. >>>> =20 >>>> Sent from my iPad >>>> =20 >>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>> wrote: >>>> =20 >>>>> Oh I see now. >>>>> =20 >>>>> It's not that the processor reorders in any tricky way. It is that the >>>>> write performed by Text8.New could be sitting around processor 1's cach= >>> e >>>>> indefinitely without actually getting written back to main memory. >>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>> inted >>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>> but >>>>> maybe not X86 (without further reorderings anyhow)? >>>>> =20 >>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>> he >>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>> e >>>>> system for a while. >>>>> =20 >>>>> Mika >>>>> =20 >>>>> =20 >>>>> Jay K writes: >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> The processor can reorder. Across call/ret at least in general. >>>>>> =20 >>>>>> =20 >>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>> ion=3D2C =3D >>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>> "LTO")=3D >>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>> =20 >>>>>> =20 >>>>>> - Jay >>>>>> =20 >>>>>> =20 >>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>> caltech.=3D >>>>>> edu >>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>> From: mika at async.caltech.edu >>>>>>> =3D20 >>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>> =3D20 >>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>> =3D20 >>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>> right=3D2C though. But it doesn't... >>>>>>> =3D20 >>>>>>> Am I missing something? >>>>>>> =3D20 >>>>>>> Mika >>>>>>> =3D20 >>>>>>> Mika Nystrom writes: >>>>>>>> =20 >>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>> you g=3D >>>>>> et >>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>> But no >>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>> n >>>>>>>> you call it with the same CHAR... >>>>>>>> =20 >>>>>>>> Mika >>>>>>>> =20 >>>>>>>> Jay K writes: >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>> * 1-char t=3D >>>>>> exts *) >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> BEGIN >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> It should be: >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> BEGIN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> RETURN fromCharCache [c] >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> END FromChar=3D3D3B >>>>>>>>> =3D3D0A=3D3D >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>> fro=3D >>>>>> mCha=3D3D >>>>>>>>> rCache[c] is made. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Can the compiler somehow catch these? >>>>>>>>> I fear they are a small epidemic. >>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>> oces=3D >>>>>> sor =3D3D >>>>>>>>> could reorder. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>> I think we do. >>>>>>>>> =20 >>>>>>>>> =20 >>>>>>>>> - Jay >>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>> =20 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>
We have race con= >>> ditions=3D >>>>>> =3D3D2C like =3D3D >>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>> 3D ARRAY=3D >>>>>> CHAR OF=3D3D >>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>> omChar =3D >>>>>> (c: CHAR=3D3D >>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>> r> =3D3D=3D >>>>>> 3B BEGIN
=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
&=3D >>>>>> nbsp=3D3D3B =3D3D >>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>> 3B
 =3D3D3B=3D >>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>> 3D3B
&nbs=3D >>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>> URN fromCharCa=3D >>>>>> che [c]
&nb=3D3D >>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>> ROCEDURE=3D >>>>>> FromCha=3D3D >>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>> IL THEN
=3D >>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>> ] :=3D3D3D c=3D3D3=3D >>>>>> B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>> =3D3D3D Text8.N=3D >>>>>> ew (buf) DO
&=3D3D >>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B =3D3D3B M=3D >>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>> p=3D3D3B =3D3D=3D >>>>>> 3B fromCharCache[c=3D3D >>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>> 3D3B END=3D3D3B>>>>>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>> =3D3D0A=3D3D >>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>> nmen=3D >>>>>> t to=3D3D >>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>> th=3D >>>>>> ese?=3D3D >>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>> n't =3D >>>>>> real=3D3D >>>>>>>>> ize where all the compiler and processor could reorder.


= >>> Do w=3D >>>>>> e ha=3D3D >>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>> r>>>>>> r> -=3D3D >>>>>>>>> Jay
>>>>>>>>> =3D3D >>>>>>>>> =20 >>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> =20 >>>>>> >>>>>> >>>>>> >>>>>>
The processor can reord= >>> er. Acros=3D >>>>>> s call/ret at least in general.


The compiler can also reord= >>> er=3D2C=3D >>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>> oft "L=3D >>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>> world sy=3D >>>>>> stems certainly do.)


 =3D3B- Jay


>> d=3D3D"Sk=3D >>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>> el at elegos=3D >>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>> race con=3D >>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>> :32:48 -0=3D >>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>> Sorry=3D2C I =3D >>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>> t Text8.N=3D >>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>> t8.New =3D >>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>> has to =3D >>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>> Yes pos=3D >>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>> but th=3D >>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>> n until=3D >>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>> runni=3D >>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>> know co=3D >>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>
&g=3D >>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>> be
=3D >>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>> B Am I missin=3D >>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>> 3B Mika Nystr=3D >>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>> ondition=3D2C i=3D >>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>> ong copy=3D >>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>> 3B >=3D3B=3D >>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>> br>&g=3D >>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>> br>>=3D3B >=3D >>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>> =3D3B>=3D3BCon=3D >>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>> 3B>=3D3BConte=3D >>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>> >=3D3B >=3D >>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>> c/Text.m3:>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>> t=3D3B>=3D3BVAR fromC=3D >>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>> ts *)
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>> AY [0..0] OF =3D >>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>> t=3D3B IF fromCh=3D >>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>> :=3D3D3D c=3D3D3=3D >>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>> w (buf)=3D3D3B<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>> B RETURN fromCha=3D >>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3B=3D >>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>> :
>=3D3B >=3D >>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>> BPROCEDURE FromChar (=3D >>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B VA=3D >>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>> =3D3D
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>> =3D3B >=3D3B>=3D3B =3D >>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>> D0A=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>> >=3D3B >=3D >>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>> 3B fromCha=3D >>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>> B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>> br>>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>> =3D3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B
>=3D3B &=3D >>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>> ssignment=3D >>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>> =3D3B >=3D3B>=3D >>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>> compiler somehow c=3D >>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>> ic.
>=3D >>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>> the compile=3D >>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>> t=3D3B >=3D3B&g=3D >>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>> have the right cons=3D >>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>> br>>=3D3B &=3D >>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>> =3D3B - Jay
>=3D3B &=3D >>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>> ent-Type: t=3D >>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>> tent-Transfer-E=3D >>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D >>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>> >=3D3B >=3D3B>=3D3B=3D >>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>> e P
>=3D3B >=3D >>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>> 3B >=3D3B>=3D3Bpadd=3D >>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>> .hmmessage
>=3D >>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>> 3D3B
>=3D3B >=3D >>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>> =3D3B >=3D3B>=3D3B=3D >>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>> have race cond=3D >>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>> e/src/Text.m3:<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>> =3D3D3D ARRAY CHAR =3D >>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>> AR=3D3D
>=3D3B=3D >>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>> buf: ARRAY [0..0]=3D >>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>> t=3D3B=3D3D
>=3D >>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>> Bnbsp=3D3D3B IF fromCha=3D >>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>> =3D3Bnbsp=3D3D
&g=3D >>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B fromCharCac=3D >>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>> 3B<=3D3Bbr>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>> arCache [c]<=3D3B=3D >>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>> romChar=3D3D3B<=3D3Bb=3D >>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>> t=3D3B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>> =3D3Br (c: CHAR): T =3D >>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>> >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>> =3D3D3D NIL THEN&l=3D >>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>> =3D3D3D c=3D3D3B<=3D >>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>> 3D3D Text8.New (buf=3D >>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>> >=3D3B>=3D3B] :=3D >>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>> 3B >=3D3B>=3D3B&=3D >>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>> 3B>=3D3B&=3D3Bnb=3D >>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>> B >=3D3B>=3D3B<=3D >>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>> ishes before =3D >>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>> s made.<=3D3B=3D >>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>> catch these?=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>> small epidemic.&l=3D >>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>> t=3D3B>=3D3Bize=3D >>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>> lt=3D3Bbr&=3D >>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>> the right construct=3D >>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>> B<=3D3Bbr>=3D >>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>> Bbr>=3D3B =3D >>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>> <=3D3B/html>=3D3B=3D >>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>> 896e-68ee-44da-82dd=3D >>>>>> -311cd8c58979_--
>>>>>> =3D >>>>>> =20 >>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>> Content-Type: text/html; charset="iso-8859-1" >>> Content-Transfer-Encoding: quoted-printable >>> >>> >>> >>> >>>
It is NOT benign.
The global = >>> can be written before the initialization of what it points to is done.
T= >>> he result is wrong.


It IS benign if multiple threads go and allo= >>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>
 =3B- Jay


&g= >>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>> r than benign? It's a cache and there's no problem if different threads get= >>> different texts. Don't cripple performance unnecessarily. The collector w= >>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>> ly without actually getting written back to main memory.
>=3B >=3B P= >>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>> long time people didn't realize where all the compiler and proces=3D
&g= >>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>> v> >>> = >>> >>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- > From hosking at cs.purdue.edu Tue Aug 28 05:21:40 2012 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Aug 2012 23:21:40 -0400 Subject: [M3devel] race conditions in lock-free code... In-Reply-To: References: <20120827042351.141B81A207D@async.async.caltech.edu> <20120827053248.A1B3F1A207D@async.async.caltech.edu> <20120827071744.781621A207D@async.async.caltech.edu> <3FFDB3F1-208D-469B-96EF-E508456F0ED8@cs.purdue.edu> <20120827183451.976971A2094@async.async.caltech.edu> <270C9CA5-6FB3-4DAF-ACAE-F645A4B02B31@cs.purdue.edu> Message-ID: <80443A80-9E57-4994-B0EF-B31C5FC98EB5@cs.purdue.edu> Indeed, a lock is overkill. In general any constructor (initializer) will need a StoreStore barrier (see http://g.oswego.edu/dl/jmm/cookbook.html) before returning a pointer to the constructed storage. Sent from my iPad On Aug 27, 2012, at 10:45 PM, Jay wrote: > Alpha may be declining in relevance but MIPS and SPARC are still around and ARM is widespread & growing. SPARC is TSO. So not necessary. StoreStore = nop. ARM needs dmb-st. Alpha needs wmb. I don't know what a StoreStore barrier looks like on MIPS. > > A lock is much more than is needed. Just a "fence" or "barrier" before the write into the global like I showed should suffice. > > Compiler should warn...? For writes to globals in the absence of locks or fences/barriers? Including knowing the call graph... Overkill. Don't scare the horses unnecessarily. Concurrent programmers must always beware. > > - Jay (briefly/pocket-sized-computer-aka-phone) > > On Aug 27, 2012, at 5:40 PM, Tony Hosking wrote: > >> On Intel we should be OK: x86-TSO causes the write to x^ to appear before the write to global on all cores. >> >> Yes, I think it may have been problematic on Alpha and may still be on PowerPC. To make the character write visible one would need a fence after the assignment to x^ before returning x. >> >> It would be straightforward to fix this using a LOCK statement around the allocation and initialization. Or alternatively, if we assume all constructors include an implicit fence then using a constructor would do the trick. >> >> Antony Hosking | Associate Professor | Computer Science | Purdue University >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Mobile +1 765 427 5484 >> >> >> >> >> >> On Aug 27, 2012, at 2:34 PM, Mika Nystrom wrote: >> >>> >>> yeah I agree. >>> >>> processor 1 performs... >>> >>> x := alloc_new_text >>> x^ := character >>> return x >>> global := x >>> >>> processor 2 performs >>> >>> IF global # NIL THEN use(global) END >>> >>> now if out of all this, x^ := character is held back, e.g., in a processor >>> write buffer, and if use(global) involves performing global^, the value >>> of global^ will be the value of global^ just after x := alloc_new_text >>> and before x^ := character. >>> >>> I believe very aggressive architectures such as Alpha permit reordering >>> to that extent. If memory serves, on Alpha, if you do not perform a >>> Memory Barrier, pretty much nothing can be inferred on one processor >>> about the ordering of operations on another. >>> >>> X86 I am not sure is that aggressive. In a hardware-synchronized >>> multiprocessor environment without barrier operations using e.g. the >>> Illinois/MESI protocol (doesn't X86?), at the point x^ := character is >>> performed, the cache coherence protocol has to guarantee the cache line >>> is in state Exclusive, which means it has to be evicted from the cache >>> in processor 2 and when use(global) comes around the protocol will >>> have to ensure it is in Shared state on processor 2. (Or Exclusive, >>> if P1 writes it back and invalidates.) This would guarantee that you >>> could not actually see a very old value for global^ at the same time >>> as a newer value for global. The protocol guarantees that the relative >>> ordering of writes is maintained across processors. >>> >>> Of course I don't know whether Alphas are actually that aggressive in >>> practice. I'd suspect that a multiprocessor 21264 could be. But I do >>> agree it's a bug relative to the Alpha Architecture Manual. A very nasty >>> one at that. >>> >>> Mika >>> >>> >>> Jay K writes: >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/plain; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> It is NOT benign. >>>> The global can be written before the initialization of what it points to is= >>>> done. >>>> The result is wrong. >>>> >>>> >>>> It IS benign if multiple threads go and allocate separate multiple initiali= >>>> zed TEXTs. >>>> That's not the problem. >>>> >>>> >>>> - Jay >>>> >>>> >>>>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com >>>>> From: hosking at cs.purdue.edu >>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400 >>>>> To: mika at async.caltech.edu >>>>> =20 >>>>> So why is this anything other than benign? It's a cache and there's no pr= >>>> oblem if different threads get different texts. Don't cripple performance u= >>>> nnecessarily. The collector will reclaim any unreachable text. >>>>> =20 >>>>> Sent from my iPad >>>>> =20 >>>>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom = >>>> wrote: >>>>> =20 >>>>>> Oh I see now. >>>>>> =20 >>>>>> It's not that the processor reorders in any tricky way. It is that the >>>>>> write performed by Text8.New could be sitting around processor 1's cach= >>>> e >>>>>> indefinitely without actually getting written back to main memory. >>>>>> Processor 2 might see the new pointer=2C but when reading the memory po= >>>> inted >>>>>> to=2C would get uninitialized memory. I think this can happen on Alpha= >>>> but >>>>>> maybe not X86 (without further reorderings anyhow)? >>>>>> =20 >>>>>> Any chance any of the pthreads bugs could be of this nature? How are t= >>>> he >>>>>> pthreads doing? I gathered some bugs were fixed but I haven't tried th= >>>> e >>>>>> system for a while. >>>>>> =20 >>>>>> Mika >>>>>> =20 >>>>>> =20 >>>>>> Jay K writes: >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> The processor can reorder. Across call/ret at least in general. >>>>>>> =20 >>>>>>> =20 >>>>>>> The compiler can also reorder=3D2C if you have whole-program-optimizat= >>>> ion=3D2C =3D >>>>>>> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc= >>>> "LTO")=3D >>>>>>> . (We don't=3D2C but real world systems certainly do.) >>>>>>> =20 >>>>>>> =20 >>>>>>> - Jay >>>>>>> =20 >>>>>>> =20 >>>>>>>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.= >>>> caltech.=3D >>>>>>> edu >>>>>>>> Subject: Re: [M3devel] race conditions in lock-free code... >>>>>>>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700 >>>>>>>> From: mika at async.caltech.edu >>>>>>>> =3D20 >>>>>>>> Sorry=3D2C I can't seem to quote your message. >>>>>>>> =3D20 >>>>>>>> You say that Text8.New's result can be assigned to fromCharCache[c] >>>>>>>> before Text8.New is done running. How is that possible? Surely >>>>>>>> Text8.New has to calculate its result before it can be assigned >>>>>>>> anywhere. Yes possibly FromChar allocates space for the result of >>>>>>>> Text8.New but the assignment of whatever appears in that space can't >>>>>>>> happen until *after* Text8.New has run. Unless you are saying that >>>>>>>> the running of Text8.New and FromChar are intertwined somehow..? >>>>>>>> I know compilers do some odd things but that doesn't sound right... >>>>>>>> =3D20 >>>>>>>> If Text8.New took fromCharCache[c] as a VAR parameter you might be >>>>>>>> right=3D2C though. But it doesn't... >>>>>>>> =3D20 >>>>>>>> Am I missing something? >>>>>>>> =3D20 >>>>>>>> Mika >>>>>>>> =3D20 >>>>>>>> Mika Nystrom writes: >>>>>>>>> =20 >>>>>>>>> Yeah it's a race condition=3D2C in theory. But is it important? If= >>>> you g=3D >>>>>>> et >>>>>>>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. = >>>> But no >>>>>>>>> one ever said FromChar was guaranteed to return the same pointer whe= >>>> n >>>>>>>>> you call it with the same CHAR... >>>>>>>>> =20 >>>>>>>>> Mika >>>>>>>>> =20 >>>>>>>>> Jay K writes: >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (= >>>> * 1-char t=3D >>>>>>> exts *) >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> BEGIN >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> It should be: >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> BEGIN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> buf [0] :=3D3D3D c=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> WITH a =3D3D3D Text8.New (buf) DO >>>>>>>>>> MemoryBarrier()=3D3D3B >>>>>>>>>> fromCharCache[c] :=3D3D3D a=3D3D3B >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> RETURN fromCharCache [c] >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> END FromChar=3D3D3B >>>>>>>>>> =3D3D0A=3D3D >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> to ensure that all of Text8.New() finishes before the assignment to= >>>> fro=3D >>>>>>> mCha=3D3D >>>>>>>>>> rCache[c] is made. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Can the compiler somehow catch these? >>>>>>>>>> I fear they are a small epidemic. >>>>>>>>>> For a long time people didn't realize where all the compiler and pr= >>>> oces=3D >>>>>>> sor =3D3D >>>>>>>>>> could reorder. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> Do we have the right constructs by now to fix them? >>>>>>>>>> I think we do. >>>>>>>>>> =20 >>>>>>>>>> =20 >>>>>>>>>> - Jay >>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_ >>>>>>>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1" >>>>>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>>>>> =20 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>
We have race con= >>>> ditions=3D >>>>>>> =3D3D2C like =3D3D >>>>>>>>>> this=3D3D2C m3core/src/Text.m3:


VAR fromCharCache :=3D3D= >>>> 3D ARRAY=3D >>>>>>> CHAR OF=3D3D >>>>>>>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)


PROCEDURE Fr= >>>> omChar =3D >>>>>>> (c: CHAR=3D3D >>>>>>>>>> ): T =3D3D3D
 =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B>>> r> =3D3D=3D >>>>>>> 3B BEGIN
=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
&=3D >>>>>>> nbsp=3D3D3B =3D3D >>>>>>>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D= >>>> 3B
 =3D3D3B=3D >>>>>>>  =3D3D3B =3D3D3B&n=3D3D >>>>>>>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D= >>>> 3D3B
&nbs=3D >>>>>>> p=3D3D3B =3D3D3B=3D3D >>>>>>>>>>  =3D3D3B END=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B RET= >>>> URN fromCharCa=3D >>>>>>> che [c]
&nb=3D3D >>>>>>>>>> sp=3D3D3B END FromChar=3D3D3B


It should be:


P= >>>> ROCEDURE=3D >>>>>>> FromCha=3D3D >>>>>>>>>> r (c: CHAR): T =3D3D3D
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B BEGIN
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N= >>>> IL THEN
=3D >>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0= >>>> ] :=3D3D3D c=3D3D3=3D >>>>>>> B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a= >>>> =3D3D3D Text8.N=3D >>>>>>> ew (buf) DO
&=3D3D >>>>>>>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B =3D3D3B M=3D >>>>>>> emoryBarrier()=3D3D3B<=3D3D >>>>>>>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs= >>>> p=3D3D3B =3D3D=3D >>>>>>> 3B fromCharCache[c=3D3D >>>>>>>>>> ] :=3D3D3D a=3D3D3B
 =3D3D3B =3D3D3B =3D3D3B =3D= >>>> 3D3B END=3D3D3B>>>>>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]
= >>>> =3D3D0A=3D3D >>>>>>>>>>  =3D3D3B END FromChar=3D3D3B
=3D3D0A=3D3D >>>>>>>>>>

to ensure that all of Text8.New() finishes before the assig= >>>> nmen=3D >>>>>>> t to=3D3D >>>>>>>>>> fromCharCache[c] is made.


Can the compiler somehow catch= >>>> th=3D >>>>>>> ese?=3D3D >>>>>>>>>>
I fear they are a small epidemic.
For a long time people did= >>>> n't =3D >>>>>>> real=3D3D >>>>>>>>>> ize where all the compiler and processor could reorder.


= >>>> Do w=3D >>>>>>> e ha=3D3D >>>>>>>>>> ve the right constructs by now to fix them?
I think we do.
>>> r>>>>>>> r> -=3D3D >>>>>>>>>> Jay
>>>>>>>>>> =3D3D >>>>>>>>>> =20 >>>>>>>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_-- >>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_ >>>>>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> =20 >>>>>>> >>>>>>> >>>>>>> >>>>>>>
The processor can reord= >>>> er. Acros=3D >>>>>>> s call/ret at least in general.


The compiler can also reord= >>>> er=3D2C=3D >>>>>>> if you have whole-program-optimization=3D2C link-time-code-gen (Micros= >>>> oft "L=3D >>>>>>> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real= >>>> world sy=3D >>>>>>> stems certainly do.)


 =3D3B- Jay


>>> d=3D3D"Sk=3D >>>>>>> yDrivePlaceholder">
>=3D3B To: jay.krell at cornell.edu=3D3B m3dev= >>>> el at elegos=3D >>>>>>> oft.com=3D3B mika at async.caltech.edu
>=3D3B Subject: Re: [M3devel]= >>>> race con=3D >>>>>>> ditions in lock-free code...
>=3D3B Date: Sun=3D2C 26 Aug 2012 22= >>>> :32:48 -0=3D >>>>>>> 700
>=3D3B From: mika at async.caltech.edu
>=3D3B
>=3D3B = >>>> Sorry=3D2C I =3D >>>>>>> can't seem to quote your message.
>=3D3B
>=3D3B You say tha= >>>> t Text8.N=3D >>>>>>> ew's result can be assigned to fromCharCache[c]
>=3D3B before Tex= >>>> t8.New =3D >>>>>>> is done running. How is that possible? Surely
>=3D3B Text8.New = >>>> has to =3D >>>>>>> calculate its result before it can be assigned
>=3D3B anywhere. = >>>> Yes pos=3D >>>>>>> sibly FromChar allocates space for the result of
>=3D3B Text8.New= >>>> but th=3D >>>>>>> e assignment of whatever appears in that space can't
>=3D3B happe= >>>> n until=3D >>>>>>> *after* Text8.New has run. Unless you are saying that
>=3D3B the= >>>> runni=3D >>>>>>> ng of Text8.New and FromChar are intertwined somehow..?
>=3D3B I = >>>> know co=3D >>>>>>> mpilers do some odd things but that doesn't sound right...
>=3D3B= >>>>
&g=3D >>>>>>> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might= >>>> be
=3D >>>>>>> >=3D3B right=3D2C though. But it doesn't...
>=3D3B
>=3D3= >>>> B Am I missin=3D >>>>>>> g something?
>=3D3B
>=3D3B Mika
>=3D3B
>=3D= >>>> 3B Mika Nystr=3D >>>>>>> om writes:
>=3D3B >=3D3B
>=3D3B >=3D3BYeah it's a race c= >>>> ondition=3D2C i=3D >>>>>>> n theory. But is it important? If you get
>=3D3B >=3D3Bthe wr= >>>> ong copy=3D >>>>>>> =3D2C there will be two TEXTs representing one CHAR. But no
>=3D= >>>> 3B >=3D3B=3D >>>>>>> one ever said FromChar was guaranteed to return the same pointer when<= >>>> br>&g=3D >>>>>>> t=3D3B >=3D3Byou call it with the same CHAR...
>=3D3B >=3D3B<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B Mika
>=3D3B >=3D3B
>=3D3B >=3D3BJay K writes:<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >= >>>> =3D3B>=3D3BCon=3D >>>>>>> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D= >>>> 3B>=3D3BConte=3D >>>>>>> nt-Transfer-Encoding: quoted-printable
>=3D3B >=3D3B>=3D3B>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr= >>>> c/Text.m3:>>>>>>> >=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B &g= >>>> t=3D3B>=3D3BVAR fromC=3D >>>>>>> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex= >>>> ts *)
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3BPROCEDURE Fr=3D >>>>>>> omChar (c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B VAR buf: ARR= >>>> AY [0..0] OF =3D >>>>>>> CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B BEGIN
>=3D3B >=3D3B&g= >>>> t=3D3B IF fromCh=3D >>>>>>> arCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B buf [0]= >>>> :=3D3D3D c=3D3D3=3D >>>>>>> B
>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne= >>>> w (buf)=3D3D3B<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B END=3D3D3B
>=3D3B >=3D3B>=3D3= >>>> B RETURN fromCha=3D >>>>>>> rCache [c]
>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3B=3D >>>>>>>
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BIt should be= >>>> :
>=3D3B >=3D >>>>>>> =3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3= >>>> BPROCEDURE FromChar (=3D >>>>>>> c: CHAR): T =3D3D3D
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B VA=3D >>>>>>> R buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3D3B >=3D3B>=3D3B=3D3D0A= >>>> =3D3D
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B BEGIN
>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D
>= >>>> =3D3B >=3D3B>=3D3B =3D >>>>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>=3D3B >=3D3B>=3D3B=3D3= >>>> D0A=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B
>=3D3B >= >>>> =3D3B>=3D3B=3D3D0A=3D3D<=3D >>>>>>> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO>>>> >=3D3B >=3D >>>>>>> =3D3B>=3D3B MemoryBarrier()=3D3D3B
>=3D3B >=3D3B>=3D= >>>> 3B fromCha=3D >>>>>>> rCache[c] :=3D3D3D a=3D3D3B
>=3D3B >=3D3B>=3D3B END=3D3D3= >>>> B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END=3D3D3B>>> r>>=3D3B >=3D3B>=3D3B=3D >>>>>>> =3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<= >>>> br>>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B END FromChar= >>>> =3D3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a= >>>> ssignment=3D >>>>>>> to fromCha=3D3D
>=3D3B >=3D3B>=3D3BrCache[c] is made.
>= >>>> =3D3B >=3D3B>=3D >>>>>>> =3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BCan the= >>>> compiler somehow c=3D >>>>>>> atch these?
>=3D3B >=3D3B>=3D3BI fear they are a small epidem= >>>> ic.
>=3D >>>>>>> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all = >>>> the compile=3D >>>>>>> r and processor =3D3D
>=3D3B >=3D3B>=3D3Bcould reorder.
&g= >>>> t=3D3B >=3D3B&g=3D >>>>>>> t=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3BDo we = >>>> have the right cons=3D >>>>>>> tructs by now to fix them?
>=3D3B >=3D3B>=3D3BI think we do.<= >>>> br>>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>= >>>> =3D3B - Jay
>=3D3B &=3D >>>>>>> gt=3D3B>=3D3B =3D3D
>=3D3B >=3D3B>= >>>> =3D3B
>=3D3B >=3D3B>=3D3B--_=3D >>>>>>> 5f23896e-68ee-44da-82dd-311cd8c58979_
>=3D3B >=3D3B>=3D3BCont= >>>> ent-Type: t=3D >>>>>>> ext/html=3D3B charset=3D3D"iso-8859-1"
>=3D3B >=3D3B>=3D3BCon= >>>> tent-Transfer-E=3D >>>>>>> ncoding: quoted-printable
>=3D3B >=3D3B>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D >>>>>>> =3D3Bhtml>=3D3B
>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B
= >>>> >=3D3B >=3D3B>=3D3B=3D >>>>>>> <=3D3Bstyle>=3D3B<=3D3B!--
>=3D3B >=3D3B>=3D3B.hmmessag= >>>> e P
>=3D3B >=3D >>>>>>> =3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B
>=3D= >>>> 3B >=3D3B>=3D3Bpadd=3D >>>>>>> ing:0px
>=3D3B >=3D3B>=3D3B}
>=3D3B >=3D3B>=3D3Bbody= >>>> .hmmessage
>=3D >>>>>>> =3D3B >=3D3B>=3D3B{
>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D= >>>> 3D3B
>=3D3B >=3D >>>>>>> =3D3B>=3D3Bfont-family:Calibri
>=3D3B >=3D3B>=3D3B}
>= >>>> =3D3B >=3D3B>=3D3B=3D >>>>>>> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B
>=3D3B >= >>>> =3D3B>=3D3B<=3D3Bbod=3D >>>>>>> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe= >>>> have race cond=3D >>>>>>> itions=3D3D2C like =3D3D
>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor= >>>> e/src/Text.m3:<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache := >>>> =3D3D3D ARRAY CHAR =3D >>>>>>> OF=3D3D
>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH= >>>> AR=3D3D
>=3D3B=3D >>>>>>> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR= >>>> buf: ARRAY [0..0]=3D >>>>>>> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g= >>>> t=3D3B=3D3D
>=3D >>>>>>> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3= >>>> Bnbsp=3D3D3B IF fromCha=3D >>>>>>> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&= >>>> =3D3Bnbsp=3D3D
&g=3D >>>>>>> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B buf =3D >>>>>>> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb= >>>> sp=3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bn=3D3D
>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B fromCharCac=3D >>>>>>> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D= >>>> 3B<=3D3Bbr>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh= >>>> arCache [c]<=3D3B=3D >>>>>>> br>=3D3B&=3D3Bnb=3D3D
>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F= >>>> romChar=3D3D3B<=3D3Bb=3D >>>>>>> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g= >>>> t=3D3B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D
>=3D3B >=3D3B>= >>>> =3D3Br (c: CHAR): T =3D >>>>>>> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&= >>>> =3D3Bnbsp=3D3D3B VAR buf:=3D >>>>>>> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B = >>>> >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >= >>>> =3D3B>=3D3B&=3D3Bnbsp=3D >>>>>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] = >>>> =3D3D3D NIL THEN&l=3D >>>>>>> t=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B=3D >>>>>>> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] := >>>> =3D3D3D c=3D3D3B<=3D >>>>>>> =3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D= >>>> 3D3D Text8.New (buf=3D >>>>>>> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D
>=3D3B >=3D3B>=3D3Bnbsp= >>>> =3D3D3B&=3D3Bnbsp=3D3D3=3D >>>>>>> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B= >>>> nbsp=3D3D3B&=3D3Bnbs=3D >>>>>>> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D
>=3D3B >=3D3B>= >>>> =3D3Bbr>=3D3B&=3D3Bn=3D >>>>>>> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B= >>>> &=3D3Bnbsp=3D3D3B&am=3D >>>>>>> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D
>=3D3B= >>>> >=3D3B>=3D3B] :=3D >>>>>>> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D= >>>> 3D3B&=3D3Bnbsp=3D3D3B&=3D >>>>>>> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D= >>>> 3B >=3D3B>=3D3B&=3D >>>>>>> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<= >>>> =3D3Bbr>=3D3B=3D3D0A=3D3D=3D >>>>>>>
>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&= >>>> amp=3D3Bnbsp=3D3D3B RET=3D >>>>>>> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3B >=3D= >>>> 3B>=3D3B&=3D3Bnb=3D >>>>>>> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D
>=3D3= >>>> B >=3D3B>=3D3B<=3D >>>>>>> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin= >>>> ishes before =3D >>>>>>> the assignment to=3D3D
>=3D3B >=3D3B>=3D3B fromCharCache[c] i= >>>> s made.<=3D3B=3D >>>>>>> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow= >>>> catch these?=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a = >>>> small epidemic.&l=3D >>>>>>> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D
>=3D3B &g= >>>> t=3D3B>=3D3Bize=3D >>>>>>> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&= >>>> lt=3D3Bbr&=3D >>>>>>> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D
>=3D3B >=3D3B>=3D3Bve = >>>> the right construct=3D >>>>>>> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3= >>>> B<=3D3Bbr>=3D >>>>>>> =3D3B<=3D3Bbr>=3D3B -=3D3D
>=3D3B >=3D3B>=3D3B Jay<=3D3= >>>> Bbr>=3D3B =3D >>>>>>> <=3D3B/div>=3D3B<=3D3B/body>=3D3B
>=3D3B >=3D3B>=3D3B= >>>> <=3D3B/html>=3D3B=3D >>>>>>> =3D3D
>=3D3B >=3D3B>=3D3B
>=3D3B >=3D3B>=3D3B--_5f23= >>>> 896e-68ee-44da-82dd=3D >>>>>>> -311cd8c58979_--
>>>>>>> =3D >>>>>>> =20 >>>>>>> --_86f22d85-5540-4565-bf73-232aaafdc94b_-- >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_ >>>> Content-Type: text/html; charset="iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>>
It is NOT benign.
The global = >>>> can be written before the initialization of what it points to is done.
T= >>>> he result is wrong.


It IS benign if multiple threads go and allo= >>>> cate separate multiple initialized TEXTs.
That's not the problem.
>>>>
 =3B- Jay


&g= >>>> t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>=3B From: hos= >>>> king at cs.purdue.edu
>=3B Subject: Re: [M3devel] race conditions in lock= >>>> -free code...
>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>=3B T= >>>> o: mika at async.caltech.edu
>=3B
>=3B So why is this anything othe= >>>> r than benign? It's a cache and there's no problem if different threads get= >>>> different texts. Don't cripple performance unnecessarily. The collector w= >>>> ill reclaim any unreachable text.
>=3B
>=3B Sent from my iPad>>>> >=3B
>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi= >>>> ka at async.caltech.edu>=3B wrote:
>=3B
>=3B >=3B Oh I see now.= >>>>
>=3B >=3B
>=3B >=3B It's not that the processor reorders in= >>>> any tricky way. It is that the
>=3B >=3B write performed by Text8.= >>>> New could be sitting around processor 1's cache
>=3B >=3B indefinite= >>>> ly without actually getting written back to main memory.
>=3B >=3B P= >>>> rocessor 2 might see the new pointer=2C but when reading the memory pointed= >>>>
>=3B >=3B to=2C would get uninitialized memory. I think this can h= >>>> appen on Alpha but
>=3B >=3B maybe not X86 (without further reorderi= >>>> ngs anyhow)?
>=3B >=3B
>=3B >=3B Any chance any of the pthre= >>>> ads bugs could be of this nature? How are the
>=3B >=3B pthreads do= >>>> ing? I gathered some bugs were fixed but I haven't tried the
>=3B >= >>>> =3B system for a while.
>=3B >=3B
>=3B >=3B Mika
>= >>>> =3B >=3B
>=3B >=3B
>=3B >=3B Jay K writes:
>=3B >= >>>> =3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>=3B >=3B>=3B C= >>>> ontent-Type: text/plain=3B charset=3D"iso-8859-1"
>=3B >=3B>=3B Co= >>>> ntent-Transfer-Encoding: quoted-printable
>=3B >=3B>=3B
>=3B= >>>> >=3B>=3B The processor can reorder. Across call/ret at least in genera= >>>> l.
>=3B >=3B>=3B
>=3B >=3B>=3B
>=3B >=3B>=3B T= >>>> he compiler can also reorder=3D2C if you have whole-program-optimization=3D= >>>> 2C =3D
>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li= >>>> nk-time-optimization (gcc "LTO")=3D
>=3B >=3B>=3B . (We don't=3D2C= >>>> but real world systems certainly do.)
>=3B >=3B>=3B
>=3B &g= >>>> t=3B>=3B
>=3B >=3B>=3B - Jay
>=3B >=3B>=3B
>=3B = >>>> >=3B>=3B
>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m= >>>> 3devel at elegosoft.com=3D3B mika at async.caltech.=3D
>=3B >=3B>=3B edu= >>>>
>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc= >>>> k-free code...
>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3= >>>> 2:48 -0700
>=3B >=3B>=3B>=3B From: mika at async.caltech.edu
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Sorry=3D2C I can'= >>>> t seem to quote your message.
>=3B >=3B>=3B>=3B =3D20
>=3B = >>>> >=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC= >>>> harCache[c]
>=3B >=3B>=3B>=3B before Text8.New is done running. = >>>> How is that possible? Surely
>=3B >=3B>=3B>=3B Text8.New has t= >>>> o calculate its result before it can be assigned
>=3B >=3B>=3B>= >>>> =3B anywhere. Yes possibly FromChar allocates space for the result of
&= >>>> gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i= >>>> n that space can't
>=3B >=3B>=3B>=3B happen until *after* Text8.= >>>> New has run. Unless you are saying that
>=3B >=3B>=3B>=3B the r= >>>> unning of Text8.New and FromChar are intertwined somehow..?
>=3B >= >>>> =3B>=3B>=3B I know compilers do some odd things but that doesn't sound = >>>> right...
>=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B If= >>>> Text8.New took fromCharCache[c] as a VAR parameter you might be
>=3B = >>>> >=3B>=3B>=3B right=3D2C though. But it doesn't...
>=3B >=3B&g= >>>> t=3B>=3B =3D20
>=3B >=3B>=3B>=3B Am I missing something?
&g= >>>> t=3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika
>= >>>> =3B >=3B>=3B>=3B =3D20
>=3B >=3B>=3B>=3B Mika Nystrom writ= >>>> es:
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B = >>>> Yeah it's a race condition=3D2C in theory. But is it important? If you g= >>>> =3D
>=3B >=3B>=3B et
>=3B >=3B>=3B>=3B>=3B the wrong = >>>> copy=3D2C there will be two TEXTs representing one CHAR. But no
>=3B = >>>> >=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th= >>>> e same pointer when
>=3B >=3B>=3B>=3B>=3B you call it with the= >>>> same CHAR...
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>=3B>= >>>> =3B>=3B Mika
>=3B >=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B Jay K writes:
>=3B >=3B>=3B>=3B>=3B>=3B --_5= >>>> f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>=3B >=3B= >>>> >=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B
>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A= >>>> RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D
>=3B >=3B>=3B= >>>> exts *)
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh= >>>> ar (c: CHAR): T =3D3D3D
>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A= >>>> RRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN>>> r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL = >>>> THEN
>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex= >>>> t8.New (buf)=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B>>> r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B It should be:
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B= >>>> >=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B BEGIN
&g= >>>> t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B= >>>> buf [0] :=3D3D3D c=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D= >>>> 3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8= >>>> .New (buf) DO
>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(= >>>> )=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] := >>>> =3D3D3D a=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B END=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B =3D3= >>>> D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c= >>>> ]
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B END FromChar=3D3D3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B = >>>> >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B to= >>>> ensure that all of Text8.New() finishes before the assignment to fro=3D>>>> >=3B >=3B>=3B mCha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B rCa= >>>> che[c] is made.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B&= >>>> gt=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Can the c= >>>> ompiler somehow catch these?
>=3B >=3B>=3B>=3B>=3B>=3B I fea= >>>> r they are a small epidemic.
>=3B >=3B>=3B>=3B>=3B>=3B For a= >>>> long time people didn't realize where all the compiler and proces=3D
&g= >>>> t=3B >=3B>=3B sor =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B could = >>>> reorder.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B&g= >>>> t=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r= >>>> ight constructs by now to fix them?
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B I think we do.
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B>=3B - Jay= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>=3B&g= >>>> t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_
>=3B >=3B>=3B>=3B&= >>>> gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B
>=3B >=3B>=3B>=3B>= >>>> =3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh= >>>> ead>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!= >>>> --
>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B {
>=3B >=3B>=3B>=3B>=3B>=3B margin:0p= >>>> x=3D3D3B
>=3B >=3B>=3B>=3B>=3B>=3B padding:0px
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B }
>=3B >=3B>=3B>=3B>=3B>=3B body= >>>> .hmmessage
>=3B >=3B>=3B>=3B>=3B>=3B {
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B font-size: 12pt=3D3D3B
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B font-family:Calibri
>=3B >=3B>=3B>=3B>=3B>=3B }>>> r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he= >>>> ad>=3B
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h= >>>> mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D>>>> >=3B >=3B>=3B =3D3D2C like =3D3D
>=3B >=3B>=3B>=3B>=3B&= >>>> gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g= >>>> t=3BVAR fromCharCache :=3D3D3D ARRAY=3D
>=3B >=3B>=3B CHAR OF=3D3D= >>>>
>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha= >>>> r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<= >>>> br>>=3B >=3B>=3B (c: CHAR=3D3D
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C= >>>> HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B BEGIN= >>>> <=3Bbr>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T= >>>> HEN<=3Bbr>=3B&=3B=3D
>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a= >>>> mp=3Bnbsp=3D3D3B=3D
>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bn=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>= >>>> =3B&=3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr= >>>> >=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha= >>>> rCa=3D
>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D
>=3B= >>>> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B= >>>> <=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B= >>>> br>=3BPROCEDURE=3D
>=3B >=3B>=3B FromCha=3D3D
>=3B >=3B&g= >>>> t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D
= >>>> >=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.= >>>> .0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>=3B&g= >>>> t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D
>=3B >= >>>> =3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D
>=3B &g= >>>> t=3B>=3B =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs= >>>> p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp= >>>> =3D3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B >=3B>=3B B<=3Bbr>=3B= >>>> =3D3D0A=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am= >>>> p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH = >>>> a =3D3D3D Text8.N=3D
>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&= >>>> =3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D= >>>> 3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3= >>>> B&=3Bnbsp=3D3D3B M=3D
>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a= >>>> mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D
>=3B >=3B>=3B 3B fromCharCache[c= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&= >>>> gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D= >>>> 3B END=3D3D3B<=3Bbr=3D
>=3B >=3B>=3B>=3B =3D3D0A=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&= >>>> =3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B&= >>>> gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE= >>>> TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D
>=3B >=3B>=3B>= >>>> =3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A= >>>> =3D3D
>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B= >>>> to ensure that all of Text8.New() finishes before the assignmen=3D
>= >>>> =3B >=3B>=3B t to=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B fromCha= >>>> rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile= >>>> r somehow catch th=3D
>=3B >=3B>=3B ese?=3D3D
>=3B >=3B>= >>>> =3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B= >>>> br>=3BFor a long time people didn't =3D
>=3B >=3B>=3B real=3D3D<= >>>> br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro= >>>> cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D
&= >>>> gt=3B >=3B>=3B e ha=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B ve th= >>>> e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&= >>>> gt=3B<=3Bbr>=3B<=3Bb=3D
>=3B >=3B>=3B r>=3B -=3D3D
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B = >>>> <=3B/div>=3B<=3B/body>=3B
>=3B >=3B>=3B>=3B>=3B>= >>>> =3B <=3B/html>=3B=3D3D
>=3B >=3B>=3B>=3B>=3B>=3B
>= >>>> =3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_= >>>> --
>=3B >=3B>=3B =3D
>=3B >=3B>= >>>> =3B
>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_
>= >>>> =3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"
>= >>>> =3B >=3B>=3B Content-Transfer-Encoding: quoted-printable
>=3B >= >>>> =3B>=3B
>=3B >=3B>=3B <=3Bhtml>=3B
>=3B >=3B>=3B &= >>>> lt=3Bhead>=3B
>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--
>=3B= >>>> >=3B>=3B .hmmessage P
>=3B >=3B>=3B {
>=3B >=3B>=3B = >>>> margin:0px=3D3B
>=3B >=3B>=3B padding:0px
>=3B >=3B>=3B }= >>>>
>=3B >=3B>=3B body.hmmessage
>=3B >=3B>=3B {
>=3B &= >>>> gt=3B>=3B font-size: 12pt=3D3B
>=3B >=3B>=3B font-family:Calibri= >>>>
>=3B >=3B>=3B }
>=3B >=3B>=3B -->=3B<=3B/style>=3B= >>>> <=3B/head>=3B
>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g= >>>> t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D
>= >>>> =3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>= >>>> =3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D
>=3B >=3B>= >>>> =3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso= >>>> ft "L=3D
>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO= >>>> "). (We don't=3D2C but real world sy=3D
>=3B >=3B>=3B stems certai= >>>> nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<= >>>> =3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk= >>>> =3D
>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg= >>>> t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D
>=3B >=3B>= >>>> =3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject= >>>> : Re: [M3devel] race con=3D
>=3B >=3B>=3B ditions in lock-free cod= >>>> e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<= >>>> br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal= >>>> tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2= >>>> C I =3D
>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>= >>>> =3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D
&= >>>> gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B before Text8.New =3D
>=3B >=3B>=3B is done run= >>>> ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New = >>>> has to =3D
>=3B >=3B>=3B calculate its result before it can be ass= >>>> igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D
>=3B >=3B>= >>>> =3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt= >>>> =3D3B Text8.New but th=3D
>=3B >=3B>=3B e assignment of whatever a= >>>> ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D
&= >>>> gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B the runni=3D
>=3B >=3B>=3B ng of Text8.= >>>> New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k= >>>> now co=3D
>=3B >=3B>=3B mpilers do some odd things but that doesn'= >>>> t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D
&= >>>> gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param= >>>> eter you might be<=3Bbr>=3B=3D
>=3B >=3B>=3B &=3Bgt=3D3B ri= >>>> ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>= >>>> =3B&=3Bgt=3D3B Am I missin=3D
>=3B >=3B>=3B g something?<=3Bb= >>>> r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&= >>>> amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D
>=3B >=3B&g= >>>> t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D
>=3B &g= >>>> t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D
>=3B >=3B>=3B =3D2C there = >>>> will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B=3D
>=3B >=3B>=3B one ever said FromChar was guara= >>>> nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D
>=3B &g= >>>> t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g= >>>> t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<= >>>> br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f= >>>> 23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3BCon=3D
>=3B >=3B>=3B tent-Type: text/plain=3D3B= >>>> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3BConte=3D
>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab= >>>> le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have= >>>> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D
= >>>> >=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B= >>>> br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D
>=3B >=3B>=3B harCa= >>>> che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<= >>>> =3Bbr>=3B&=3Bg=3D
>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg= >>>> t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D
>=3B >= >>>> =3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D
>=3B >=3B>=3B CH= >>>> AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l= >>>> t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D
= >>>> >=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D
>=3B &= >>>> gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B = >>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D
>=3B >=3B= >>>> >=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D= >>>>
>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B = >>>> >=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC= >>>> EDURE FromChar (=3D
>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D
>=3B >=3B>=3B R bu= >>>> f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B &= >>>> gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt= >>>> =3D3B&=3Bgt=3D3B =3D
>=3B >=3B>=3B IF fromCharCache [c] =3D3D= >>>> 3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A= >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D
>=3B >=3B>=3B br&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.= >>>> New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>= >>>> =3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg= >>>> t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D
>=3B >=3B>= >>>> =3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
&g= >>>> t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&= >>>> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c= >>>> ]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>= >>>> =3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3= >>>> B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >=3B>=3B = >>>> gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the = >>>> assignment=3D
>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D
>=3B >=3B>= >>>> =3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f= >>>> ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B&g= >>>> t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz= >>>> e where all the compile=3D
>=3B >=3B>=3B r and processor =3D3D<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D
>=3B >=3B>=3B t=3D= >>>> 3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a= >>>> mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D
&g= >>>> t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a= >>>> mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &= >>>> =3B=3D
>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D
>=3B >= >>>> =3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B= >>>> &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am= >>>> p=3Bgt=3D3B&=3Bgt=3D3B--_=3D
>=3B >=3B>=3B 5f23896e-68ee-44da-8= >>>> 2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC= >>>> ontent-Type: t=3D
>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885= >>>> 9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf= >>>> er-E=3D
>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B= >>>> gt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bhtml&=3Bgt= >>>> =3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3= >>>> Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B=3D
>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B= >>>> !--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<= >>>> =3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Bg= >>>> t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px= >>>> =3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D
= >>>> >=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm= >>>> message<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf= >>>> ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B &g= >>>> t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3B=3D
>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty= >>>> le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D= >>>> 3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D
>=3B >=3B>=3B = >>>> y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am= >>>> p=3Bgt=3D3BWe have race cond=3D
>=3B >=3B>=3B itions=3D3D2C like = >>>> =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m= >>>> 3core/src/Text.m3:&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac= >>>> he :=3D3D3D ARRAY CHAR =3D
>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&= >>>> =3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char = >>>> texts *)&=3Blt=3D3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt= >>>> =3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C= >>>> HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D
>=3B >=3B>=3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D3D3B VAR buf: ARRAY [0..0]=3D
>=3B >=3B>=3B OF CHAR=3D3D3B&am= >>>> p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr= >>>> &=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D
>=3B >=3B>=3B =3D3B = >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D
>=3B >=3B>=3B rCac= >>>> he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D
>=3B >=3B>= >>>> =3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D
>=3B >=3B&= >>>> gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B&g= >>>> t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3= >>>> B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D
>= >>>> =3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D
>=3B >=3B= >>>> >=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B= >>>> amp=3D
>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am= >>>> p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D
>=3B= >>>> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt= >>>> =3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D= >>>> 3Bb=3D
>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> &=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a= >>>> mp=3Blt=3D3Bbr&=3Bgt=3D
>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >>>> &=3Bgt=3D3Br (c: CHAR): T =3D
>=3B >=3B>=3B =3D3D3D&=3Blt=3D= >>>> 3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a= >>>> mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D
>=3B >=3B>=3B A= >>>> RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
>=3B &= >>>> gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb= >>>> sp=3D
>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D
>=3B &g= >>>> t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D= >>>> 3D3B=3D
>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs= >>>> p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D>>>> >=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B= >>>> gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam= >>>> p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf= >>>> =3D
>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3= >>>> B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&= >>>> amp=3Bamp=3D3Bnbsp=3D3D3=3D
>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbs=3D
>=3B >=3B>=3B p=3D3D3B MemoryBarrier(= >>>> )=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&= >>>> =3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D
>=3B >=3B>=3B bsp=3D= >>>> 3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp= >>>> =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D
>=3B >=3B>=3B p=3D3B= >>>> nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am= >>>> p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D
>=3B >=3B>=3B =3D3D= >>>> 3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba= >>>> mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D
>=3B >=3B>= >>>> =3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3= >>>> D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D
= >>>> >=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D= >>>> 3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D
&g= >>>> t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B= >>>> &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3= >>>> D3B RET=3D
>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt= >>>> =3D3B&=3Bamp=3D3Bnb=3D
>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3= >>>> D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &= >>>> amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D
>=3B >=3B>=3B =3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f= >>>> inishes before =3D
>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am= >>>> p=3Blt=3D3B=3D
>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&= >>>> =3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes= >>>> e?=3D
>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D= >>>> 3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide= >>>> mic.&=3Bl=3D
>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim= >>>> e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B= >>>> gt=3D3Bize=3D
>=3B >=3B>=3B where all the compiler and processor c= >>>> ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D
&g= >>>> t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb= >>>> r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D= >>>>
>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B= >>>> I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D>>>> >=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>= >>>> =3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt= >>>> =3D3B =3D
>=3B >=3B>=3B &=3Blt=3D3B/div= >>>> &=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B= >>>> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D
>=3B= >>>> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3= >>>> B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-= >>>> 44da-82dd=3D
>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di= >>>> v>=3B <=3B/div>=3B<=3B/body>=3B
>=3B= >>>> >=3B>=3B <=3B/html>=3B=3D
>=3B >=3B>=3B
>=3B >=3B= >>>> >=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--
>>> v> >>>> = >>>> >>>> --_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_-- >> From dabenavidesd at yahoo.es Wed Aug 29 18:08:28 2012 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 29 Aug 2012 17:08:28 +0100 (BST) Subject: [M3devel] Copyright clearance in Modula-3 books Message-ID: <1346256508.9030.YahooMailClassic@web29702.mail.ird.yahoo.com> Hi all: how about asking for any part involved copyright clearance (compliance) with respect of SPwM3, M3 and AiM3. I knew of recent efforts here: http://thid.thesa.com/thid-0698-8201-th-1129-8613 Shouldn't we ask Greg Nelson for that? In that case we wouldn't need direct authorization to publish the Book, but just that, else stay tuned for a licensing of the material for us. Anyway if we have the sources of the book and nobody's claim that, couldn't be already cleared? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: