From rodney.bates at wichita.edu Tue Nov 4 15:45:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 04 Nov 2008 08:45:35 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <4910600F.7000009@wichita.edu> I have new and greatly expanded documentation on m3gdb. It is created in docbook format, and I have managed to get it to convert to one-page html. I intend to do multi-page html and pdf also. I am not sure where in the repository this should go, where to put html files so they are available on the web, or what to do with mechanically derived files in the repository. I thought of creating a new subdirectory doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary file, not a directory. Also, I presume that Olaf will want to get the html to look like the other pages, which some kind of style sheet should do, but I don't know where to find that either. Any help will be appreciated. Rodney Bates From wagner at elegosoft.com Tue Nov 4 17:04:32 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 04 Nov 2008 17:04:32 +0100 Subject: [M3devel] New m3gdb documentation In-Reply-To: <4910600F.7000009@wichita.edu> References: <4910600F.7000009@wichita.edu> Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com> Quoting "Rodney M. Bates" : > I have new and greatly expanded documentation on m3gdb. It is created > in docbook format, and I have managed to get it to convert to one-page > html. I intend to do multi-page html and pdf also. All documentation is checked-in below cm3/doc and cm3/www. m3gdb should probably be integrated into doc. > I am not sure where in the repository this should go, where to put html > files so they are available on the web, or what to do with mechanically > derived files in the repository. I thought of creating a new subdirectory > doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary > file, not a directory. just use `cvs add ' > Also, I presume that Olaf will want to get the html to look like the other > pages, which some kind of style sheet should do, but I don't know where to > find that either. The style sheet is normal.css at the top level. You can reference it like this in the HTML header: It's checked in at cm3/www/normal.css. When you've checked-in all your additions, we need to ship them to the WWW server to make them visible. I can do that if you like, just let me know when you're finished. It's great that you're taking care of m3gdb and thus providing a working debugger for all M3 users! Regards, Olaf > Any help will be appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney.bates at wichita.edu Thu Nov 6 17:31:23 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 06 Nov 2008 10:31:23 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <49131BDB.20903@wichita.edu> There is a new and greatly expanded web page with documentation on m3gdb, the Modula-3 debugger, located at: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html It's also accessible with more indexing, through the normal Modula-3 web pages, for example, starting at: http://modula3.elegosoft.com/cm3 and clicking through: -> Documentation -> CM3 Operations Guide-> m3gdb Rodney M. Bates From jay.krell at cornell.edu Sat Nov 8 16:40:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:40:50 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Message-ID: Tony, how about switching these #if's? I explain some of why below. parse.c: static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The first code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } You should be able to repro this by configuring m3cg for mips64-openbsd. I've only tested on cygwin host, but I could check others.. Thanks, - Jay From jay.krell at cornell.edu Sat Nov 8 16:45:15 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:45:15 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? Message-ID: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From hosking at cs.purdue.edu Sat Nov 8 23:55:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 8 Nov 2008 22:55:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 02:25:32 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 01:25:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: AMD64_LINUX_RTLinker.mc Type: application/octet-stream Size: 17622 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MIPS64_OPENBSD_RTLinker.mc Type: application/octet-stream Size: 17692 bytes Desc: not available URL: From hosking at cs.purdue.edu Sun Nov 9 11:01:58 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 10:01:58 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 12:10:54 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 11:10:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 15:09:10 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 14:09:10 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen Message-ID: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: ****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Nov 9 19:02:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 18:02:41 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 09:28:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 08:28:18 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()=VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END;END F1; BEGIN F1();END t. Darn. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails:****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 10 12:13:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 10 Nov 2008 11:13:05 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: > @M3paranoidgc always succeeds. I see, reading the code, it only does > reads. Nevertheless. > > Simply programs that just allocate memory forever also succeed. > Such as this: > > jay at amd64a:/dev2/cm3/t$ cat src/t.m3 > MODULE t EXPORTS Main; > TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; > TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; > > VAR a: REFANY; > > PROCEDURE F1()= > VAR b: REFANY; > BEGIN > WHILE TRUE DO > b := NEW(REF T2); > a := NEW(REF T1); > END; > END F1; > > BEGIN > F1(); > END t. > > Darn. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 18:02:41 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] AMD64_LINUX gc/stubgen > > > Clearly a problem with GC. I'm guessing you have something wrong > (pointer alignment?) in the configuration. I suspect if you run > with @M3paranoidgc you'll trip over the problem. > > On 9 Nov 2008, at 14:09, Jay wrote: > > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 20:35:19 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 19:35:19 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Message-ID: It /seems/ like any switch or switch combination makes it stop happening. Admittedly I haven't run a loop just a few runs by hand for each combo. Maybe have it record the allocation quantities and "play them back"? - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Mon, 10 Nov 2008 11:13:05 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3 MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()= VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END; END F1; BEGIN F1(); END t. Darn. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 18:02:41 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: *** *** runtime error: *** NEW() was unable to allocate more memory. *** file "../src/runtime/common/RTAllocator.m3", line 285 *** Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay From rcoleburn at scires.com Tue Nov 11 00:17:31 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Mon, 10 Nov 2008 18:17:31 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> Message-ID: <491879AF.1E75.00D7.1@scires.com> Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy >>> Jay Krell 11/9/2008 1:26 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/11/09 07:20:15 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files: cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message: begin combining nt386.common and unix.common (into cm3cg.common) mips64_openbsd support being migrating config-no-install/solgnu and solsun to Unix.common (had this sitting around, in intermediate state) go back to old formating because that is all the old C written m3build could handle add probing for host/target specific cm3cg (so I can stop copying it around as I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Nov 11 02:06:07 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 11 Nov 2008 01:06:07 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <491879AF.1E75.00D7.1@scires.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> <491879AF.1E75.00D7.1@scires.com> Message-ID: If you use my config files, yes. Note that there are two of them, one just includes the other. Take "the other". This is just because on my system, I think: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common NT386 is the oddball for me, since for any other targets, for me, it goes: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common I'd like to move NT386* to config-no-install, but I avoided doing so so far. I'd like to replace config with config-no-install, but avoid that too. - Jay Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy>>> Jay Krell 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Nov 11 21:39:09 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 11 Nov 2008 14:39:09 -0600 Subject: [M3devel] Still can't bootstrap cm3 Message-ID: <4919ED6D.6000700@wichita.edu> We still have serious problems with the cm3 system building. I have again gotten into a big hampster wheel of several different failures, and still can't rebuild a working compiler. I have gone through enough steps that I am not just positive I have remembered it all right, but it was something like this: I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and ./install-cm3-compiler.sh upgrade. That left me with a compiler that always failed with: ------------------------------------------------------------------------------------------------------------ rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime error: undefined variable: ROOT --procedure-- -line- -file--- _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Then I tried rebuilding using various do-cm3-* scripts, including after supplying argument realclean (which, I have learned many times in the past, is essential to avoid all sorts of problems), with the same results. Somewhere in here, I started getting the bad version stamp failures too. Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, a package at a time, I was consistently getting: -------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: unable to open "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading --procedure-- -line- -file--- import_version -- import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Perhaps it was the realclean that deleted all the .M3EXPORTS files. ../boot_cm3-core.sh LINUXLIBC6 gave this failure: ----------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh please specify a cross compilation target platform rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6 /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ --- building in LINUXLIBC6 --- new source -> compiling RTHooks.i3 /bin/sh: /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found m3_backend => 127 m3cc (aka cm3cg) failed compiling: RTHooks.ic --------------------------------------------------------------------------------------------------------- Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed on all the Modula-3 code). I got around the above by putting in a symbolic link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6, so it could find cm3cg. Then I went through the package-at-a-time order Tony had given in an old post to the m3devel list. That had other problems. Running cm3 in the package directory failed with the undefined ROOT problem, but do-pkg.sh would do the build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in each package with cm3 -ship. After building and shipping m3 middle and m3front, package cm3 itself wouldn't link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3, and m3core. I skipped it and m3gc*, which don't exist now, and went through the rest of the order, which completed without failures building. But that gave me a compiler that fails with a new problem I have never seen before: --------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical *** *** runtime error: *** A compile-time type is missing. *** file "../src/Utils.m3" *** Aborted (core dumped) cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 /usr/local/cm3/bin/cm3- cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg /usr/local/cm3/bin/cm3cg- cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg ------------------------------------------------------------------------------------------------------------ Going back to an older cm3 makes that go away and makes the ROOT problem recur. Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3" get the compiler to run, but it doesn't do anything, because it thinks nothing needs to be recompiled. I could delete stuff, but I fear that would just start me around another iteration of the same thing. I don't believe I have ever gotten a bootstrap to work. I think I have always had to download a recent binary and go from there. Any help would be greatly appreciated. Rodney Bates From jay.krell at cornell.edu Wed Nov 12 05:04:44 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 04:04:44 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py. ROOT should be set in the config file, or as you showed, on the command line. upgrade.sh/upgrade.py will figure it out and set it on the command line. I'd kind of like the compiler to set it itself, but that is kind of inappropriate. It's definitely possible that starting from too old of a release won't work. Avoid any script with "boot" in the name. Those are I believe for bringing up new targets and esp. cross building, not for updating mature working systems that can build natively. The dash in LINUXLIBC6-LINUXLIBC6 confirms it. The dash appears when the host and target are different. The first platform is the host, the second is the target. - Jay From wagner at elegosoft.com Wed Nov 12 08:18:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 12 Nov 2008 08:18:55 +0100 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Quoting "Rodney M. Bates" : > We still have serious problems with the cm3 system building. > I have again gotten into a big hampster wheel of several different > failures, and still can't rebuild a working compiler. > I have gone through enough steps that I am not just positive > I have remembered it all right, but it was something like this: Rodney, it's not easy to understand what has gone wrong at this point. Usually, if you have a working system, cm3/scripts/upgrade.sh should be able to bootstrap to a new set of sources. This is tested every day in the tinderbox release tests, too. As the last builds seem to have succeeded, I don't think there is a real problem in our code base currently. You may want to have a look at test_build_current in cm3/scripts/regression/defs.sh to see what's done in the regression builds. I'd suggest you either start again from the last release or one of the latest snapshots available from the build machine. I assume you are not building a cross compiler, but just trying to perform an upgrade. Daily snapshots are available from http://www.opencm3.net/snaps/snapshot-index.html I hope this helps, Olaf > I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and > ./install-cm3-compiler.sh upgrade. That left me with a compiler > that always failed with: > > ------------------------------------------------------------------------------------------------------------ > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake > runtime error: undefined variable: ROOT > > --procedure-- -line- -file--- > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Then I tried rebuilding using various do-cm3-* scripts, including after > supplying argument realclean (which, I have learned many times in the > past, is essential to avoid all sorts of problems), with the same results. > Somewhere in here, I started getting the bad version stamp failures too. > Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, > a package at a time, I was consistently getting: > > -------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime > error: unable to open > "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" > for reading > > --procedure-- -line- -file--- > import_version -- > import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Perhaps it was the realclean that deleted all the .M3EXPORTS files. > ../boot_cm3-core.sh LINUXLIBC6 gave this failure: > > ----------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > please specify a cross compilation target platform > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > LINUXLIBC6 > /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build > -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 > sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 > m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph > parseparams realgeometry set slisp sortedtableextras table-list > tempfiles > === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === > +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ > --- building in LINUXLIBC6 --- > > new source -> compiling RTHooks.i3 > /bin/sh: > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not > found > m3_backend => 127 > m3cc (aka cm3cg) failed compiling: RTHooks.ic > > --------------------------------------------------------------------------------------------------------- > > Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed > on all the Modula-3 code). I got around the above by putting in a symbolic > link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to > cm3/m3-sys/m3cc/LINUXLIBC6, > so it could find cm3cg. > > Then I went through the package-at-a-time order Tony had given in an old post > to the m3devel list. That had other problems. Running cm3 in the package > directory failed with the undefined ROOT problem, but do-pkg.sh would do the > build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in > each package with cm3 -ship. > > After building and shipping m3 middle and m3front, package cm3 > itself wouldn't > link, wanting libraries for m3back, m3objfile, m3quake, sysutils, > m3linker, libm3, > and m3core. I skipped it and m3gc*, which don't exist now, and went > through the > rest of the order, which completed without failures building. But that gave > me a compiler that fails with a new problem I have never seen before: > > --------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ > ./install-cm3-compiler.sh upgrade > cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical > cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical > > > *** > *** runtime error: > *** A compile-time type is missing. > *** file "../src/Utils.m3" > *** > > Aborted (core dumped) > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 > /usr/local/cm3/bin/cm3- > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg > /usr/local/cm3/bin/cm3cg- > cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 > cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg > > ------------------------------------------------------------------------------------------------------------ > > Going back to an older cm3 makes that go away and makes the ROOT problem > recur. Working around that with cm3 > -DROOT="/home/rodney/proj/m3/cm3-new/cm3" > get the compiler to run, but it doesn't do anything, because it thinks > nothing needs to be recompiled. I could delete stuff, but I fear that would > just start me around another iteration of the same thing. > > I don't believe I have ever gotten a bootstrap to work. I think I > have always > had to download a recent binary and go from there. > > Any help would be greatly appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Nov 12 08:57:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 07:57:31 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Nov 12 09:05:03 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 08:05:03 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow. I can only hope that optimization and/or not crashing makes it much faster. The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though. System is 200MHz+ and 256meg RAM though, ought to suffice.. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Nov 12 19:49:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 12 Nov 2008 13:49:19 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com> References: <20081112152142.02AEA10D5970@birch.elegosoft.com> Message-ID: <491ADDBA.1E75.00D7.1@scires.com> Rodney: Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?) Regards, Randy >>> Rodney M. Bates 11/12/2008 4:21 PM >>> CVSROOT:/usr/cvs Changes by:rodney at birch.08/11/12 16:21:41 Modified files: cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 Log message: Pickles in both PM3 and CM3 have some hard-coded fingerprints for common types, but the values are not consistent about byte order. Fix CM2 pickle-reading code to accept values of types NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY, written by PM3 pickle-writing code. Probably, the latter two can't even happen in a pickle, unless somebody wrote a custom special for them. This still won't handle a more complex type that depends on these types, e.g., an object that contains a field of one of them. This latter is probably not feasable, because it would require "decrypting" fingerprints, and the hash algorithm for fingerprints is lossy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Nov 12 21:46:53 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 12 Nov 2008 20:46:53 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 01:04:32 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 00:04:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Nov 13 16:45:34 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 09:45:34 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Message-ID: <491C4B9E.4010307@wichita.edu> Olaf Wagner wrote: > > it's not easy to understand what has gone wrong at this point. > Usually, if you have a working system, cm3/scripts/upgrade.sh > should be able to bootstrap to a new set of sources. This is tested > every day in the tinderbox release tests, too. As the last builds seem > to have succeeded, I don't think there is a real problem in our > code base currently. You may want to have a look at test_build_current > in cm3/scripts/regression/defs.sh to see what's done in the regression > builds. > > I'd suggest you either start again from the last release or > one of the latest snapshots available from the build machine. Yes, that's what I always end up doing. But it would be nice to be able someday to get my local compiler to rebuild itself :-(. But even that is not working for me. Reinstalling a snapshot from 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad version stamps on Compiler.Platform and Compiler.ThisPlatform, while compiling SocketPosix.m3. Going to the most recent (at the time) snapshot of 2008-11-12, do-cm3-core.sh will run without failures, but leaves a compiler that always gets the undefined ROOT failure, even trying to compile a trivial program. > I assume you are not building a cross compiler, but just trying > to perform an upgrade. That's right. The only reason I messed with the cross scripts was that I hoped they might take care of the necessary bootstrapping in a way the would work. > > Daily snapshots are available from > http://www.opencm3.net/snaps/snapshot-index.html > > I hope this helps, > > Olaf > > From jay.krell at cornell.edu Thu Nov 13 17:37:47 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:37:47 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Randy, ROOT not being defined is a function of your config files. Look at mine. Mine are willing to ferry it from an environment variable. The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required. ROOT is the root of the source tree, as you correctly?successfully worked around. You are supposed to be able to use the compiler without a source tree. But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself, but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he is saying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases. But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run, I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store, but I don't think it was worth the effort...the effort I put in to clone the .sh in .cmd. I think I punted doing that in the *.py, and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program. Hm. Could be your config file wants it? Could be my config files wants it? I guess the usual refrain -- send us your config file(s). I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 17:54:55 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:54:55 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Also, until you find something that works: export OMIT_GCC=yes Otherwise you are going to waste a lot more time experimenting. - Jay From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3 Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Nov 13 18:06:22 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 11:06:22 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu> This is odd. I've been rebuilding my local compiler by hand several times in the last few weeks. I will try the scripts and follow up. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 13 Nov 2008, at 09:45, Rodney M. Bates wrote: > Olaf Wagner wrote: >> it's not easy to understand what has gone wrong at this point. >> Usually, if you have a working system, cm3/scripts/upgrade.sh >> should be able to bootstrap to a new set of sources. This is tested >> every day in the tinderbox release tests, too. As the last builds >> seem >> to have succeeded, I don't think there is a real problem in our >> code base currently. You may want to have a look at >> test_build_current >> in cm3/scripts/regression/defs.sh to see what's done in the >> regression >> builds. >> I'd suggest you either start again from the last release or >> one of the latest snapshots available from the build machine. > > Yes, that's what I always end up doing. But it would be nice to be > able someday to get my local compiler to rebuild itself :-(. > > But even that is not working for me. Reinstalling a snapshot from > 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing > .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad > version stamps on Compiler.Platform and Compiler.ThisPlatform, > while compiling SocketPosix.m3. > > Going to the most recent (at the time) snapshot of 2008-11-12, > do-cm3-core.sh will run without failures, but leaves a compiler > that always gets the undefined ROOT failure, even trying to compile > a trivial program. > >> I assume you are not building a cross compiler, but just trying >> to perform an upgrade. > > That's right. The only reason I messed with the cross scripts was > that > I hoped they might take care of the necessary bootstrapping in a way > the would work. > >> Daily snapshots are available from >> http://www.opencm3.net/snaps/snapshot-index.html >> I hope this helps, >> Olaf >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 00:29:17 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 17:29:17 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 Message-ID: <491CB84D.4000008@wichita.edu> Here is how the undefined ROOT problem happens when trying to rebuild cm3 with itself. I had been making the assumption that commands build, ship, clean, when supplied to the do-cm3-*.sh scripts, did the same things as -build, -ship, -clean supplied to the cm3 command. Not so. build actually does cm3 -build -override, as do buildlocal and empty commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. There is a buildglobal command that does -build without -override, but it also does a ship as well. There appears to be no way to just build, without either overriding or shipping. So, I have been running do-cm3-*.sh with no argument (and sometimes build). This starts a chain of events that eventually leads to an installed libm3 that always executes quake script bundle-ov.tmpl, and it requires that either M3BUNDLE or ROOT be defined, neither of which normally is. This will happen trying to compile any program that uses libm3, which is just about everything. We should document this somewhere. From jay.krell at cornell.edu Fri Nov 14 02:44:44 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 01:44:44 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: I don't know, but something doesn't seem right. D:\cm3>dir /s/b bund* *ov* *temp* *tmp*... D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = BIN_USE & SL & "m3bundle"end D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl :\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl bundle-ov.tmp: if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & SL & "m3bundle", BUILD_DIR) M3BUNDLE = _M3BUNDLEend => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used. D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile if defined("_M3BUNDLE_OVERRIDE") template ("bundle-ov")endtemplate ("bundle") You defined _M3BUNDLE_OVERRIDE somewhere along the way? Try replacing the one file with the other.. - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 03:01:30 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 20:01:30 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: References: <491CB84D.4000008@wichita.edu> Message-ID: <491CDBFA.7030500@wichita.edu> Jay wrote: > I don't know, but something doesn't seem right. > > D:\cm3>dir /s/b bund* *ov* *temp* *tmp* > ... > D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = BIN_USE & SL & "m3bundle" > end > > D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* > D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > :\dev2\cm3.2>dir /s/b /a-d bund*tmp* > :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > bundle-ov.tmp: > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & > SL & "m3bundle", BUILD_DIR) > M3BUNDLE = _M3BUNDLE > end > > => this file requires ROOT or M3BUNDLE like you say, but this isn't > normally used. > > D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile > > if defined("_M3BUNDLE_OVERRIDE") > template ("bundle-ov") > end > template ("bundle") > > You defined _M3BUNDLE_OVERRIDE somewhere along the way? Maybe I should have given more detail on the chain of events. You got most of it. The missing piece is: m3-libs/libm3/src/m3overrides contains: _M3BUNDLE_OVERRIDE = "T" So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE, and the rest is as you have noted. > > Try replacing the one file with the other.. > > - Jay > > > > > Date: Thu, 13 Nov 2008 17:29:17 -0600 > > From: rodney.bates at wichita.edu > > To: m3devel at elegosoft.com > > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 > > > > Here is how the undefined ROOT problem happens when trying to rebuild > > cm3 with itself. > > > > I had been making the assumption that commands build, ship, clean, when > > supplied to the do-cm3-*.sh scripts, did the same things as -build, > > -ship, -clean supplied to the cm3 command. Not so. > > > > build actually does cm3 -build -override, as do buildlocal and empty > > commands to the scripts. clean does cm3 -clean -override, as does > cleanlocal. > > There is a buildglobal command that does -build without -override, but > > it also does a ship as well. There appears to be no way to just build, > > without either overriding or shipping. > > > > So, I have been running do-cm3-*.sh with no argument (and sometimes > build). > > This starts a chain of events that eventually leads to an installed libm3 > > that always executes quake script bundle-ov.tmpl, and it requires that > > either M3BUNDLE or ROOT be defined, neither of which normally is. This > > will happen trying to compile any program that uses libm3, which is just > > about everything. > > > > We should document this somewhere. > From hosking at cs.purdue.edu Fri Nov 14 03:50:07 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 20:50:07 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CDBFA.7030500@wichita.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> I generally run with buildship for these commands. On 13 Nov 2008, at 20:01, Rodney M. Bates wrote: > Jay wrote: >> I don't know, but something doesn't seem right. >> D:\cm3>dir /s/b bund* *ov* *temp* *tmp* >> ... >> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = BIN_USE & SL & "m3bundle" >> end >> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> :\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> bundle-ov.tmp: >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & >> "%s" & >> SL & "m3bundle", BUILD_DIR) >> M3BUNDLE = _M3BUNDLE >> end >> => this file requires ROOT or M3BUNDLE like you say, but this isn't >> normally used. >> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile >> if defined("_M3BUNDLE_OVERRIDE") >> template ("bundle-ov") >> end >> template ("bundle") >> You defined _M3BUNDLE_OVERRIDE somewhere along the way? > > Maybe I should have given more detail on the chain of events. > You got most of it. The missing piece is: > > m3-libs/libm3/src/m3overrides contains: > _M3BUNDLE_OVERRIDE = "T" > > So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the > above file of quake code to be interpreted, defining > _M3BUNDLE_OVERRIDE, > and the rest is as you have noted. > >> Try replacing the one file with the other.. >> - Jay >> > Date: Thu, 13 Nov 2008 17:29:17 -0600 >> > From: rodney.bates at wichita.edu >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 >> > >> > Here is how the undefined ROOT problem happens when trying to >> rebuild >> > cm3 with itself. >> > >> > I had been making the assumption that commands build, ship, >> clean, when >> > supplied to the do-cm3-*.sh scripts, did the same things as -build, >> > -ship, -clean supplied to the cm3 command. Not so. >> > >> > build actually does cm3 -build -override, as do buildlocal and >> empty >> > commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> > There is a buildglobal command that does -build without - >> override, but >> > it also does a ship as well. There appears to be no way to just >> build, >> > without either overriding or shipping. >> > >> > So, I have been running do-cm3-*.sh with no argument (and >> sometimes build). >> > This starts a chain of events that eventually leads to an >> installed libm3 >> > that always executes quake script bundle-ov.tmpl, and it requires >> that >> > either M3BUNDLE or ROOT be defined, neither of which normally is. >> This >> > will happen trying to compile any program that uses libm3, which >> is just >> > about everything. >> > >> > We should document this somewhere. From wagner at elegosoft.com Fri Nov 14 09:35:07 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 14 Nov 2008 09:35:07 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. Perhaps you could write up some paragraphs for the scripts' README and the web pages? I always seem to overlook or tend to obscure some crucial points for other users :-/ Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Fri Nov 14 12:08:17 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 11:08:17 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: > From: hosking> > I generally run with buildship for these commands. That, plus if you "buildlocal", you are supposed to be barred from "shipping". That is, if you build with -override, you aren't supposed to be able to "ship". "ship" is to copy from the source tree to the install tree. I think I did see a problem once where it let me ship "local" stuff. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 16:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Fri, 14 Nov 2008 09:26:05 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: <491D988D.9080804@wichita.edu> Tony Hosking wrote: > I generally run with buildship for these commands. And so will I, henceforth. Rodney Bates From jay.krell at cornell.edu Fri Nov 14 21:24:55 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 20:24:55 +0000 Subject: [M3devel] Linux/amd64 Message-ID: I turned of gc just while running stubgen. Leading to: http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz Please try them? See how well they work for you all? I haven't run most of the stuff, but cm3 does build itself. gc does usually?often work. (saying "posix" I feel is redundant...) - Jay From jay.krell at cornell.edu Sun Nov 16 16:31:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 15:31:25 +0000 Subject: [M3devel] duplicated code? Message-ID: Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails either there or soon thereafter. The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 16 17:18:40 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 16:18:40 +0000 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: Er, the large allocation does come from the gc itself. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO I'll know more shortly. I guess..it looks like the heap can be discontiguous, but we do record keeping for what it all spans. The comments say: (* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff | p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5M GetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M GetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for. t1,t2,t3 are just the lengths of the strings being concatented. Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig. And it seems the heap allocator wants to allocate an array to describe the pages. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc. Perhaps perhaps blowing it up, to say, 1meg, will address this? But really, an array to describe pages spanning the results of separate memory allocations, seems wrong. A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code? Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 03:14:55 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 16 Nov 2008 20:14:55 -0600 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> On 16 Nov 2008, at 10:18, Jay wrote: > Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? No assumption. > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. Correct. > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. Ah, yes, that is most unfortunate. > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 09:39:48 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 08:39:48 +0000 Subject: [M3devel] AMD64_LINUX (again) Message-ID: ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8 ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Nov 17 10:10:43 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:10:43 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Mon Nov 17 10:30:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:30:00 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Hm, now I seem to be the victim of mail truncation. I'll try a top-quote this time (our system administrator will investigate this). Third attempt: It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Nov 17 10:35:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 09:35:18 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: 1meg pages weren't large enough to avoid running out of memory. Any thoughts on using the boehm-gc? I don't know if it is compacting. Or even generational. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level). When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Either that, or a binary searched array where you only store entries for state changes. You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 13:21:26 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 06:21:26 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: On 17 Nov 2008, at 03:35, Jay wrote: > 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? > Any thoughts on using the boehm-gc? Hell no. > I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). > Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. > Or anything about it really, just that it is much ported and > presumably well maintained, since it is part of "gcc", part of the > gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. > Hm... I assume the right data structure here is a multi level array, > that you index by picking off progressively less significant bits of > an address (or page number) just like the hardware page tables for > virtual memory (but without the perf benefits of a cache and using > physical addresses after the first level). > When an entire level is at the same state, due to skipping a large > span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. > Either that, or a binary searched array where you only store entries > for state changes. > You'd use like the STL's upper_bound/lower_bound/equal_range > functions, that is, really the same thing as bsearch, but when an > entry isn't found, you return where it would be inserted, which is > like the last place you looked before giving up. I'm concerned about the lookup costs for this. > It might be worth confirming "that I'm not crazy" -- that mmap > behaves this way for other people, or that skimming its code > suggests it is not surprising. Could be related to how malloc works > also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". > > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 16 Nov 2008 20:14:55 -0600 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] duplicated code? > > > On 16 Nov 2008, at 10:18, Jay wrote: > > Er, the large allocation does come from the gc itself. > > What's being allocated? The side array should not be too huge. > > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? > > No assumption. > > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. > > Correct. > > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. > > Ah, yes, that is most unfortunate. > > > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? > > I am working on minimizing the need for the global array, but we do > need something that can be easily indexed like this. Perhaps we > need to pass a hint to RTOS.GetMemory that will try to allocate its > regions close together. > > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. > > Indeed. As mentioned above, I am working to eliminate the need for > this. Code that starts us on this path will be checked in within a > day or so. > > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 13:48:59 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 12:48:59 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: understood, ok. I don't know the algorithm for allocating pages. There is a "hint" parameter to mmap, it isn't exactly clear how it is handled. Probably giving the block we have plus its size is reasonable. I can try that out at some point, though using sbrk seems ok. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600 On 17 Nov 2008, at 03:35, Jay wrote: 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? Any thoughts on using the boehm-gc? Hell no. I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. I'm concerned about the lookup costs for this. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 14:00:01 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 13:00:01 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: > 1 meg pages didn't suffice > Is this still the sidespan array problem? I assume so. I just tried it out, got the out of memory, and gave up on that approach. I guess I could bump it up to 1 gig or more, just to prove the theory but of course at some point I'm squeamish about wasting address space or actual memory. 1gig is tiny for address space but not for actual memory. (assuming a full 64 bit address space, which actually is probably not available by far) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 19:31:01 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 12:31:01 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: If sbrk works let's leave it at that for now. On 17 Nov 2008, at 07:00, Jay wrote: > > 1 meg pages didn't suffice > > Is this still the sidespan array problem? > > I assume so. > I just tried it out, got the out of memory, and gave up on that > approach. > I guess I could bump it up to 1 gig or more, just to prove the > theory but > of course at some point I'm squeamish about wasting address space or > actual memory. > 1gig is tiny for address space but not for actual memory. > (assuming a full 64 bit address space, which actually is probably > not available by far) > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Sun Nov 23 19:06:18 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 23 Nov 2008 12:06:18 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> References: <491CB84D.4000008@wichita.edu> <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Message-ID: <49299B9A.7050103@wichita.edu> Olaf Wagner wrote: > Hm, now I seem to be the victim of mail truncation. I'll try a top-quote > this time (our system administrator will investigate this). Third attempt: > > It seems that this time my email wasn't complete, for whatever reasons. > I was out of town for some days so I noticed only today. > > Rodney, I did ask you if you could write up some paragraphs for the > README in scripts and the web pages that clarifies this matter so that > others won't run into it again? Done. > > I always seem to miss some crucial points when trying to explain the > build system myself... > > Thanks in advance, > > Olaf > > Quoting "Rodney M. Bates" : > >> Here is how the undefined ROOT problem happens when trying to rebuild >> cm3 with itself. >> >> I had been making the assumption that commands build, ship, clean, when >> supplied to the do-cm3-*.sh scripts, did the same things as -build, >> -ship, -clean supplied to the cm3 command. Not so. >> >> build actually does cm3 -build -override, as do buildlocal and empty >> commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> There is a buildglobal command that does -build without -override, but >> it also does a ship as well. There appears to be no way to just build, >> without either overriding or shipping. >> >> So, I have been running do-cm3-*.sh with no argument (and sometimes build) From michael.anderson at elego.de Wed Nov 26 03:10:24 2008 From: michael.anderson at elego.de (Michael Anderson) Date: Wed, 26 Nov 2008 03:10:24 +0100 Subject: [M3devel] compiling cvsup on Debian Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Howdy there, I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using this thread as starting point: https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html Thus far, I've: - applied the patch - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;" - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", as well as the corresponding procedure calls as outlined here: https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html Which that leaves me with just one teensy error: "../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait) which does go away if I change the IMPORT and procedure calls back to SchedulerPosix for JUST SigHandler.m3. Is it safe to use both "Scheduler" and "SchedulerPosix", or will Bad_Things happen? Thanks, mike -- Michael Anderson IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 michael.anderson at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Nov 26 17:03:52 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 26 Nov 2008 10:03:52 -0600 Subject: [M3devel] compiling cvsup on Debian In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu> Should be just fine! On 25 Nov 2008, at 20:10, Michael Anderson wrote: > Howdy there, > > I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, > using this thread as > starting point: > https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html > > Thus far, I've: > > - applied the patch > - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF > Utime.struct_timezone;" > - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", > as well as the > corresponding procedure calls as outlined here: > > https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html > > Which that leaves me with just one teensy error: > > "../src/SigHandler.m3", line 221: unknown qualification > '.' (IOAlertWait) > > which does go away if I change the IMPORT and procedure calls back > to SchedulerPosix for > JUST SigHandler.m3. > > Is it safe to use both "Scheduler" and "SchedulerPosix", or will > Bad_Things happen? > > Thanks, > mike > > > -- > Michael Anderson > IT Services & Support > > elego Software Solutions GmbH > Gustav-Meyer-Allee 25 > Building 12.3 (BIG) room 227 > 13355 Berlin, Germany > > phone +49 30 23 45 86 96 michael.anderson at elegosoft.com > fax +49 30 23 45 86 95 http://www.elegosoft.com > > Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin > Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 > > From rodney.bates at wichita.edu Tue Nov 4 15:45:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 04 Nov 2008 08:45:35 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <4910600F.7000009@wichita.edu> I have new and greatly expanded documentation on m3gdb. It is created in docbook format, and I have managed to get it to convert to one-page html. I intend to do multi-page html and pdf also. I am not sure where in the repository this should go, where to put html files so they are available on the web, or what to do with mechanically derived files in the repository. I thought of creating a new subdirectory doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary file, not a directory. Also, I presume that Olaf will want to get the html to look like the other pages, which some kind of style sheet should do, but I don't know where to find that either. Any help will be appreciated. Rodney Bates From wagner at elegosoft.com Tue Nov 4 17:04:32 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 04 Nov 2008 17:04:32 +0100 Subject: [M3devel] New m3gdb documentation In-Reply-To: <4910600F.7000009@wichita.edu> References: <4910600F.7000009@wichita.edu> Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com> Quoting "Rodney M. Bates" : > I have new and greatly expanded documentation on m3gdb. It is created > in docbook format, and I have managed to get it to convert to one-page > html. I intend to do multi-page html and pdf also. All documentation is checked-in below cm3/doc and cm3/www. m3gdb should probably be integrated into doc. > I am not sure where in the repository this should go, where to put html > files so they are available on the web, or what to do with mechanically > derived files in the repository. I thought of creating a new subdirectory > doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary > file, not a directory. just use `cvs add ' > Also, I presume that Olaf will want to get the html to look like the other > pages, which some kind of style sheet should do, but I don't know where to > find that either. The style sheet is normal.css at the top level. You can reference it like this in the HTML header: It's checked in at cm3/www/normal.css. When you've checked-in all your additions, we need to ship them to the WWW server to make them visible. I can do that if you like, just let me know when you're finished. It's great that you're taking care of m3gdb and thus providing a working debugger for all M3 users! Regards, Olaf > Any help will be appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney.bates at wichita.edu Thu Nov 6 17:31:23 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 06 Nov 2008 10:31:23 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <49131BDB.20903@wichita.edu> There is a new and greatly expanded web page with documentation on m3gdb, the Modula-3 debugger, located at: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html It's also accessible with more indexing, through the normal Modula-3 web pages, for example, starting at: http://modula3.elegosoft.com/cm3 and clicking through: -> Documentation -> CM3 Operations Guide-> m3gdb Rodney M. Bates From jay.krell at cornell.edu Sat Nov 8 16:40:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:40:50 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Message-ID: Tony, how about switching these #if's? I explain some of why below. parse.c: static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The first code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } You should be able to repro this by configuring m3cg for mips64-openbsd. I've only tested on cygwin host, but I could check others.. Thanks, - Jay From jay.krell at cornell.edu Sat Nov 8 16:45:15 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:45:15 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? Message-ID: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From hosking at cs.purdue.edu Sat Nov 8 23:55:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 8 Nov 2008 22:55:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 02:25:32 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 01:25:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: AMD64_LINUX_RTLinker.mc Type: application/octet-stream Size: 17622 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MIPS64_OPENBSD_RTLinker.mc Type: application/octet-stream Size: 17692 bytes Desc: not available URL: From hosking at cs.purdue.edu Sun Nov 9 11:01:58 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 10:01:58 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 12:10:54 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 11:10:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 15:09:10 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 14:09:10 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen Message-ID: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: ****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Nov 9 19:02:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 18:02:41 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 09:28:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 08:28:18 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()=VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END;END F1; BEGIN F1();END t. Darn. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails:****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 10 12:13:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 10 Nov 2008 11:13:05 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: > @M3paranoidgc always succeeds. I see, reading the code, it only does > reads. Nevertheless. > > Simply programs that just allocate memory forever also succeed. > Such as this: > > jay at amd64a:/dev2/cm3/t$ cat src/t.m3 > MODULE t EXPORTS Main; > TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; > TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; > > VAR a: REFANY; > > PROCEDURE F1()= > VAR b: REFANY; > BEGIN > WHILE TRUE DO > b := NEW(REF T2); > a := NEW(REF T1); > END; > END F1; > > BEGIN > F1(); > END t. > > Darn. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 18:02:41 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] AMD64_LINUX gc/stubgen > > > Clearly a problem with GC. I'm guessing you have something wrong > (pointer alignment?) in the configuration. I suspect if you run > with @M3paranoidgc you'll trip over the problem. > > On 9 Nov 2008, at 14:09, Jay wrote: > > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 20:35:19 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 19:35:19 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Message-ID: It /seems/ like any switch or switch combination makes it stop happening. Admittedly I haven't run a loop just a few runs by hand for each combo. Maybe have it record the allocation quantities and "play them back"? - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Mon, 10 Nov 2008 11:13:05 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3 MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()= VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END; END F1; BEGIN F1(); END t. Darn. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 18:02:41 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: *** *** runtime error: *** NEW() was unable to allocate more memory. *** file "../src/runtime/common/RTAllocator.m3", line 285 *** Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay From rcoleburn at scires.com Tue Nov 11 00:17:31 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Mon, 10 Nov 2008 18:17:31 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> Message-ID: <491879AF.1E75.00D7.1@scires.com> Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy >>> Jay Krell 11/9/2008 1:26 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/11/09 07:20:15 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files: cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message: begin combining nt386.common and unix.common (into cm3cg.common) mips64_openbsd support being migrating config-no-install/solgnu and solsun to Unix.common (had this sitting around, in intermediate state) go back to old formating because that is all the old C written m3build could handle add probing for host/target specific cm3cg (so I can stop copying it around as I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Nov 11 02:06:07 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 11 Nov 2008 01:06:07 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <491879AF.1E75.00D7.1@scires.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> <491879AF.1E75.00D7.1@scires.com> Message-ID: If you use my config files, yes. Note that there are two of them, one just includes the other. Take "the other". This is just because on my system, I think: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common NT386 is the oddball for me, since for any other targets, for me, it goes: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common I'd like to move NT386* to config-no-install, but I avoided doing so so far. I'd like to replace config with config-no-install, but avoid that too. - Jay Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy>>> Jay Krell 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Nov 11 21:39:09 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 11 Nov 2008 14:39:09 -0600 Subject: [M3devel] Still can't bootstrap cm3 Message-ID: <4919ED6D.6000700@wichita.edu> We still have serious problems with the cm3 system building. I have again gotten into a big hampster wheel of several different failures, and still can't rebuild a working compiler. I have gone through enough steps that I am not just positive I have remembered it all right, but it was something like this: I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and ./install-cm3-compiler.sh upgrade. That left me with a compiler that always failed with: ------------------------------------------------------------------------------------------------------------ rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime error: undefined variable: ROOT --procedure-- -line- -file--- _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Then I tried rebuilding using various do-cm3-* scripts, including after supplying argument realclean (which, I have learned many times in the past, is essential to avoid all sorts of problems), with the same results. Somewhere in here, I started getting the bad version stamp failures too. Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, a package at a time, I was consistently getting: -------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: unable to open "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading --procedure-- -line- -file--- import_version -- import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Perhaps it was the realclean that deleted all the .M3EXPORTS files. ../boot_cm3-core.sh LINUXLIBC6 gave this failure: ----------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh please specify a cross compilation target platform rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6 /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ --- building in LINUXLIBC6 --- new source -> compiling RTHooks.i3 /bin/sh: /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found m3_backend => 127 m3cc (aka cm3cg) failed compiling: RTHooks.ic --------------------------------------------------------------------------------------------------------- Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed on all the Modula-3 code). I got around the above by putting in a symbolic link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6, so it could find cm3cg. Then I went through the package-at-a-time order Tony had given in an old post to the m3devel list. That had other problems. Running cm3 in the package directory failed with the undefined ROOT problem, but do-pkg.sh would do the build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in each package with cm3 -ship. After building and shipping m3 middle and m3front, package cm3 itself wouldn't link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3, and m3core. I skipped it and m3gc*, which don't exist now, and went through the rest of the order, which completed without failures building. But that gave me a compiler that fails with a new problem I have never seen before: --------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical *** *** runtime error: *** A compile-time type is missing. *** file "../src/Utils.m3" *** Aborted (core dumped) cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 /usr/local/cm3/bin/cm3- cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg /usr/local/cm3/bin/cm3cg- cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg ------------------------------------------------------------------------------------------------------------ Going back to an older cm3 makes that go away and makes the ROOT problem recur. Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3" get the compiler to run, but it doesn't do anything, because it thinks nothing needs to be recompiled. I could delete stuff, but I fear that would just start me around another iteration of the same thing. I don't believe I have ever gotten a bootstrap to work. I think I have always had to download a recent binary and go from there. Any help would be greatly appreciated. Rodney Bates From jay.krell at cornell.edu Wed Nov 12 05:04:44 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 04:04:44 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py. ROOT should be set in the config file, or as you showed, on the command line. upgrade.sh/upgrade.py will figure it out and set it on the command line. I'd kind of like the compiler to set it itself, but that is kind of inappropriate. It's definitely possible that starting from too old of a release won't work. Avoid any script with "boot" in the name. Those are I believe for bringing up new targets and esp. cross building, not for updating mature working systems that can build natively. The dash in LINUXLIBC6-LINUXLIBC6 confirms it. The dash appears when the host and target are different. The first platform is the host, the second is the target. - Jay From wagner at elegosoft.com Wed Nov 12 08:18:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 12 Nov 2008 08:18:55 +0100 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Quoting "Rodney M. Bates" : > We still have serious problems with the cm3 system building. > I have again gotten into a big hampster wheel of several different > failures, and still can't rebuild a working compiler. > I have gone through enough steps that I am not just positive > I have remembered it all right, but it was something like this: Rodney, it's not easy to understand what has gone wrong at this point. Usually, if you have a working system, cm3/scripts/upgrade.sh should be able to bootstrap to a new set of sources. This is tested every day in the tinderbox release tests, too. As the last builds seem to have succeeded, I don't think there is a real problem in our code base currently. You may want to have a look at test_build_current in cm3/scripts/regression/defs.sh to see what's done in the regression builds. I'd suggest you either start again from the last release or one of the latest snapshots available from the build machine. I assume you are not building a cross compiler, but just trying to perform an upgrade. Daily snapshots are available from http://www.opencm3.net/snaps/snapshot-index.html I hope this helps, Olaf > I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and > ./install-cm3-compiler.sh upgrade. That left me with a compiler > that always failed with: > > ------------------------------------------------------------------------------------------------------------ > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake > runtime error: undefined variable: ROOT > > --procedure-- -line- -file--- > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Then I tried rebuilding using various do-cm3-* scripts, including after > supplying argument realclean (which, I have learned many times in the > past, is essential to avoid all sorts of problems), with the same results. > Somewhere in here, I started getting the bad version stamp failures too. > Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, > a package at a time, I was consistently getting: > > -------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime > error: unable to open > "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" > for reading > > --procedure-- -line- -file--- > import_version -- > import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Perhaps it was the realclean that deleted all the .M3EXPORTS files. > ../boot_cm3-core.sh LINUXLIBC6 gave this failure: > > ----------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > please specify a cross compilation target platform > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > LINUXLIBC6 > /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build > -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 > sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 > m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph > parseparams realgeometry set slisp sortedtableextras table-list > tempfiles > === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === > +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ > --- building in LINUXLIBC6 --- > > new source -> compiling RTHooks.i3 > /bin/sh: > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not > found > m3_backend => 127 > m3cc (aka cm3cg) failed compiling: RTHooks.ic > > --------------------------------------------------------------------------------------------------------- > > Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed > on all the Modula-3 code). I got around the above by putting in a symbolic > link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to > cm3/m3-sys/m3cc/LINUXLIBC6, > so it could find cm3cg. > > Then I went through the package-at-a-time order Tony had given in an old post > to the m3devel list. That had other problems. Running cm3 in the package > directory failed with the undefined ROOT problem, but do-pkg.sh would do the > build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in > each package with cm3 -ship. > > After building and shipping m3 middle and m3front, package cm3 > itself wouldn't > link, wanting libraries for m3back, m3objfile, m3quake, sysutils, > m3linker, libm3, > and m3core. I skipped it and m3gc*, which don't exist now, and went > through the > rest of the order, which completed without failures building. But that gave > me a compiler that fails with a new problem I have never seen before: > > --------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ > ./install-cm3-compiler.sh upgrade > cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical > cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical > > > *** > *** runtime error: > *** A compile-time type is missing. > *** file "../src/Utils.m3" > *** > > Aborted (core dumped) > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 > /usr/local/cm3/bin/cm3- > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg > /usr/local/cm3/bin/cm3cg- > cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 > cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg > > ------------------------------------------------------------------------------------------------------------ > > Going back to an older cm3 makes that go away and makes the ROOT problem > recur. Working around that with cm3 > -DROOT="/home/rodney/proj/m3/cm3-new/cm3" > get the compiler to run, but it doesn't do anything, because it thinks > nothing needs to be recompiled. I could delete stuff, but I fear that would > just start me around another iteration of the same thing. > > I don't believe I have ever gotten a bootstrap to work. I think I > have always > had to download a recent binary and go from there. > > Any help would be greatly appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Nov 12 08:57:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 07:57:31 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Nov 12 09:05:03 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 08:05:03 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow. I can only hope that optimization and/or not crashing makes it much faster. The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though. System is 200MHz+ and 256meg RAM though, ought to suffice.. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Nov 12 19:49:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 12 Nov 2008 13:49:19 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com> References: <20081112152142.02AEA10D5970@birch.elegosoft.com> Message-ID: <491ADDBA.1E75.00D7.1@scires.com> Rodney: Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?) Regards, Randy >>> Rodney M. Bates 11/12/2008 4:21 PM >>> CVSROOT:/usr/cvs Changes by:rodney at birch.08/11/12 16:21:41 Modified files: cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 Log message: Pickles in both PM3 and CM3 have some hard-coded fingerprints for common types, but the values are not consistent about byte order. Fix CM2 pickle-reading code to accept values of types NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY, written by PM3 pickle-writing code. Probably, the latter two can't even happen in a pickle, unless somebody wrote a custom special for them. This still won't handle a more complex type that depends on these types, e.g., an object that contains a field of one of them. This latter is probably not feasable, because it would require "decrypting" fingerprints, and the hash algorithm for fingerprints is lossy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Nov 12 21:46:53 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 12 Nov 2008 20:46:53 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 01:04:32 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 00:04:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Nov 13 16:45:34 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 09:45:34 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Message-ID: <491C4B9E.4010307@wichita.edu> Olaf Wagner wrote: > > it's not easy to understand what has gone wrong at this point. > Usually, if you have a working system, cm3/scripts/upgrade.sh > should be able to bootstrap to a new set of sources. This is tested > every day in the tinderbox release tests, too. As the last builds seem > to have succeeded, I don't think there is a real problem in our > code base currently. You may want to have a look at test_build_current > in cm3/scripts/regression/defs.sh to see what's done in the regression > builds. > > I'd suggest you either start again from the last release or > one of the latest snapshots available from the build machine. Yes, that's what I always end up doing. But it would be nice to be able someday to get my local compiler to rebuild itself :-(. But even that is not working for me. Reinstalling a snapshot from 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad version stamps on Compiler.Platform and Compiler.ThisPlatform, while compiling SocketPosix.m3. Going to the most recent (at the time) snapshot of 2008-11-12, do-cm3-core.sh will run without failures, but leaves a compiler that always gets the undefined ROOT failure, even trying to compile a trivial program. > I assume you are not building a cross compiler, but just trying > to perform an upgrade. That's right. The only reason I messed with the cross scripts was that I hoped they might take care of the necessary bootstrapping in a way the would work. > > Daily snapshots are available from > http://www.opencm3.net/snaps/snapshot-index.html > > I hope this helps, > > Olaf > > From jay.krell at cornell.edu Thu Nov 13 17:37:47 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:37:47 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Randy, ROOT not being defined is a function of your config files. Look at mine. Mine are willing to ferry it from an environment variable. The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required. ROOT is the root of the source tree, as you correctly?successfully worked around. You are supposed to be able to use the compiler without a source tree. But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself, but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he is saying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases. But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run, I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store, but I don't think it was worth the effort...the effort I put in to clone the .sh in .cmd. I think I punted doing that in the *.py, and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program. Hm. Could be your config file wants it? Could be my config files wants it? I guess the usual refrain -- send us your config file(s). I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 17:54:55 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:54:55 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Also, until you find something that works: export OMIT_GCC=yes Otherwise you are going to waste a lot more time experimenting. - Jay From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3 Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Nov 13 18:06:22 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 11:06:22 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu> This is odd. I've been rebuilding my local compiler by hand several times in the last few weeks. I will try the scripts and follow up. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 13 Nov 2008, at 09:45, Rodney M. Bates wrote: > Olaf Wagner wrote: >> it's not easy to understand what has gone wrong at this point. >> Usually, if you have a working system, cm3/scripts/upgrade.sh >> should be able to bootstrap to a new set of sources. This is tested >> every day in the tinderbox release tests, too. As the last builds >> seem >> to have succeeded, I don't think there is a real problem in our >> code base currently. You may want to have a look at >> test_build_current >> in cm3/scripts/regression/defs.sh to see what's done in the >> regression >> builds. >> I'd suggest you either start again from the last release or >> one of the latest snapshots available from the build machine. > > Yes, that's what I always end up doing. But it would be nice to be > able someday to get my local compiler to rebuild itself :-(. > > But even that is not working for me. Reinstalling a snapshot from > 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing > .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad > version stamps on Compiler.Platform and Compiler.ThisPlatform, > while compiling SocketPosix.m3. > > Going to the most recent (at the time) snapshot of 2008-11-12, > do-cm3-core.sh will run without failures, but leaves a compiler > that always gets the undefined ROOT failure, even trying to compile > a trivial program. > >> I assume you are not building a cross compiler, but just trying >> to perform an upgrade. > > That's right. The only reason I messed with the cross scripts was > that > I hoped they might take care of the necessary bootstrapping in a way > the would work. > >> Daily snapshots are available from >> http://www.opencm3.net/snaps/snapshot-index.html >> I hope this helps, >> Olaf >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 00:29:17 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 17:29:17 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 Message-ID: <491CB84D.4000008@wichita.edu> Here is how the undefined ROOT problem happens when trying to rebuild cm3 with itself. I had been making the assumption that commands build, ship, clean, when supplied to the do-cm3-*.sh scripts, did the same things as -build, -ship, -clean supplied to the cm3 command. Not so. build actually does cm3 -build -override, as do buildlocal and empty commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. There is a buildglobal command that does -build without -override, but it also does a ship as well. There appears to be no way to just build, without either overriding or shipping. So, I have been running do-cm3-*.sh with no argument (and sometimes build). This starts a chain of events that eventually leads to an installed libm3 that always executes quake script bundle-ov.tmpl, and it requires that either M3BUNDLE or ROOT be defined, neither of which normally is. This will happen trying to compile any program that uses libm3, which is just about everything. We should document this somewhere. From jay.krell at cornell.edu Fri Nov 14 02:44:44 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 01:44:44 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: I don't know, but something doesn't seem right. D:\cm3>dir /s/b bund* *ov* *temp* *tmp*... D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = BIN_USE & SL & "m3bundle"end D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl :\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl bundle-ov.tmp: if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & SL & "m3bundle", BUILD_DIR) M3BUNDLE = _M3BUNDLEend => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used. D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile if defined("_M3BUNDLE_OVERRIDE") template ("bundle-ov")endtemplate ("bundle") You defined _M3BUNDLE_OVERRIDE somewhere along the way? Try replacing the one file with the other.. - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 03:01:30 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 20:01:30 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: References: <491CB84D.4000008@wichita.edu> Message-ID: <491CDBFA.7030500@wichita.edu> Jay wrote: > I don't know, but something doesn't seem right. > > D:\cm3>dir /s/b bund* *ov* *temp* *tmp* > ... > D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = BIN_USE & SL & "m3bundle" > end > > D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* > D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > :\dev2\cm3.2>dir /s/b /a-d bund*tmp* > :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > bundle-ov.tmp: > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & > SL & "m3bundle", BUILD_DIR) > M3BUNDLE = _M3BUNDLE > end > > => this file requires ROOT or M3BUNDLE like you say, but this isn't > normally used. > > D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile > > if defined("_M3BUNDLE_OVERRIDE") > template ("bundle-ov") > end > template ("bundle") > > You defined _M3BUNDLE_OVERRIDE somewhere along the way? Maybe I should have given more detail on the chain of events. You got most of it. The missing piece is: m3-libs/libm3/src/m3overrides contains: _M3BUNDLE_OVERRIDE = "T" So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE, and the rest is as you have noted. > > Try replacing the one file with the other.. > > - Jay > > > > > Date: Thu, 13 Nov 2008 17:29:17 -0600 > > From: rodney.bates at wichita.edu > > To: m3devel at elegosoft.com > > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 > > > > Here is how the undefined ROOT problem happens when trying to rebuild > > cm3 with itself. > > > > I had been making the assumption that commands build, ship, clean, when > > supplied to the do-cm3-*.sh scripts, did the same things as -build, > > -ship, -clean supplied to the cm3 command. Not so. > > > > build actually does cm3 -build -override, as do buildlocal and empty > > commands to the scripts. clean does cm3 -clean -override, as does > cleanlocal. > > There is a buildglobal command that does -build without -override, but > > it also does a ship as well. There appears to be no way to just build, > > without either overriding or shipping. > > > > So, I have been running do-cm3-*.sh with no argument (and sometimes > build). > > This starts a chain of events that eventually leads to an installed libm3 > > that always executes quake script bundle-ov.tmpl, and it requires that > > either M3BUNDLE or ROOT be defined, neither of which normally is. This > > will happen trying to compile any program that uses libm3, which is just > > about everything. > > > > We should document this somewhere. > From hosking at cs.purdue.edu Fri Nov 14 03:50:07 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 20:50:07 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CDBFA.7030500@wichita.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> I generally run with buildship for these commands. On 13 Nov 2008, at 20:01, Rodney M. Bates wrote: > Jay wrote: >> I don't know, but something doesn't seem right. >> D:\cm3>dir /s/b bund* *ov* *temp* *tmp* >> ... >> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = BIN_USE & SL & "m3bundle" >> end >> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> :\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> bundle-ov.tmp: >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & >> "%s" & >> SL & "m3bundle", BUILD_DIR) >> M3BUNDLE = _M3BUNDLE >> end >> => this file requires ROOT or M3BUNDLE like you say, but this isn't >> normally used. >> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile >> if defined("_M3BUNDLE_OVERRIDE") >> template ("bundle-ov") >> end >> template ("bundle") >> You defined _M3BUNDLE_OVERRIDE somewhere along the way? > > Maybe I should have given more detail on the chain of events. > You got most of it. The missing piece is: > > m3-libs/libm3/src/m3overrides contains: > _M3BUNDLE_OVERRIDE = "T" > > So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the > above file of quake code to be interpreted, defining > _M3BUNDLE_OVERRIDE, > and the rest is as you have noted. > >> Try replacing the one file with the other.. >> - Jay >> > Date: Thu, 13 Nov 2008 17:29:17 -0600 >> > From: rodney.bates at wichita.edu >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 >> > >> > Here is how the undefined ROOT problem happens when trying to >> rebuild >> > cm3 with itself. >> > >> > I had been making the assumption that commands build, ship, >> clean, when >> > supplied to the do-cm3-*.sh scripts, did the same things as -build, >> > -ship, -clean supplied to the cm3 command. Not so. >> > >> > build actually does cm3 -build -override, as do buildlocal and >> empty >> > commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> > There is a buildglobal command that does -build without - >> override, but >> > it also does a ship as well. There appears to be no way to just >> build, >> > without either overriding or shipping. >> > >> > So, I have been running do-cm3-*.sh with no argument (and >> sometimes build). >> > This starts a chain of events that eventually leads to an >> installed libm3 >> > that always executes quake script bundle-ov.tmpl, and it requires >> that >> > either M3BUNDLE or ROOT be defined, neither of which normally is. >> This >> > will happen trying to compile any program that uses libm3, which >> is just >> > about everything. >> > >> > We should document this somewhere. From wagner at elegosoft.com Fri Nov 14 09:35:07 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 14 Nov 2008 09:35:07 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. Perhaps you could write up some paragraphs for the scripts' README and the web pages? I always seem to overlook or tend to obscure some crucial points for other users :-/ Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Fri Nov 14 12:08:17 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 11:08:17 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: > From: hosking> > I generally run with buildship for these commands. That, plus if you "buildlocal", you are supposed to be barred from "shipping". That is, if you build with -override, you aren't supposed to be able to "ship". "ship" is to copy from the source tree to the install tree. I think I did see a problem once where it let me ship "local" stuff. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 16:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Fri, 14 Nov 2008 09:26:05 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: <491D988D.9080804@wichita.edu> Tony Hosking wrote: > I generally run with buildship for these commands. And so will I, henceforth. Rodney Bates From jay.krell at cornell.edu Fri Nov 14 21:24:55 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 20:24:55 +0000 Subject: [M3devel] Linux/amd64 Message-ID: I turned of gc just while running stubgen. Leading to: http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz Please try them? See how well they work for you all? I haven't run most of the stuff, but cm3 does build itself. gc does usually?often work. (saying "posix" I feel is redundant...) - Jay From jay.krell at cornell.edu Sun Nov 16 16:31:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 15:31:25 +0000 Subject: [M3devel] duplicated code? Message-ID: Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails either there or soon thereafter. The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 16 17:18:40 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 16:18:40 +0000 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: Er, the large allocation does come from the gc itself. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO I'll know more shortly. I guess..it looks like the heap can be discontiguous, but we do record keeping for what it all spans. The comments say: (* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff | p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5M GetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M GetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for. t1,t2,t3 are just the lengths of the strings being concatented. Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig. And it seems the heap allocator wants to allocate an array to describe the pages. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc. Perhaps perhaps blowing it up, to say, 1meg, will address this? But really, an array to describe pages spanning the results of separate memory allocations, seems wrong. A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code? Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 03:14:55 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 16 Nov 2008 20:14:55 -0600 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> On 16 Nov 2008, at 10:18, Jay wrote: > Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? No assumption. > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. Correct. > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. Ah, yes, that is most unfortunate. > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 09:39:48 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 08:39:48 +0000 Subject: [M3devel] AMD64_LINUX (again) Message-ID: ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8 ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Nov 17 10:10:43 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:10:43 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Mon Nov 17 10:30:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:30:00 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Hm, now I seem to be the victim of mail truncation. I'll try a top-quote this time (our system administrator will investigate this). Third attempt: It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Nov 17 10:35:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 09:35:18 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: 1meg pages weren't large enough to avoid running out of memory. Any thoughts on using the boehm-gc? I don't know if it is compacting. Or even generational. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level). When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Either that, or a binary searched array where you only store entries for state changes. You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 13:21:26 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 06:21:26 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: On 17 Nov 2008, at 03:35, Jay wrote: > 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? > Any thoughts on using the boehm-gc? Hell no. > I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). > Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. > Or anything about it really, just that it is much ported and > presumably well maintained, since it is part of "gcc", part of the > gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. > Hm... I assume the right data structure here is a multi level array, > that you index by picking off progressively less significant bits of > an address (or page number) just like the hardware page tables for > virtual memory (but without the perf benefits of a cache and using > physical addresses after the first level). > When an entire level is at the same state, due to skipping a large > span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. > Either that, or a binary searched array where you only store entries > for state changes. > You'd use like the STL's upper_bound/lower_bound/equal_range > functions, that is, really the same thing as bsearch, but when an > entry isn't found, you return where it would be inserted, which is > like the last place you looked before giving up. I'm concerned about the lookup costs for this. > It might be worth confirming "that I'm not crazy" -- that mmap > behaves this way for other people, or that skimming its code > suggests it is not surprising. Could be related to how malloc works > also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". > > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 16 Nov 2008 20:14:55 -0600 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] duplicated code? > > > On 16 Nov 2008, at 10:18, Jay wrote: > > Er, the large allocation does come from the gc itself. > > What's being allocated? The side array should not be too huge. > > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? > > No assumption. > > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. > > Correct. > > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. > > Ah, yes, that is most unfortunate. > > > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? > > I am working on minimizing the need for the global array, but we do > need something that can be easily indexed like this. Perhaps we > need to pass a hint to RTOS.GetMemory that will try to allocate its > regions close together. > > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. > > Indeed. As mentioned above, I am working to eliminate the need for > this. Code that starts us on this path will be checked in within a > day or so. > > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 13:48:59 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 12:48:59 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: understood, ok. I don't know the algorithm for allocating pages. There is a "hint" parameter to mmap, it isn't exactly clear how it is handled. Probably giving the block we have plus its size is reasonable. I can try that out at some point, though using sbrk seems ok. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600 On 17 Nov 2008, at 03:35, Jay wrote: 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? Any thoughts on using the boehm-gc? Hell no. I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. I'm concerned about the lookup costs for this. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 14:00:01 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 13:00:01 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: > 1 meg pages didn't suffice > Is this still the sidespan array problem? I assume so. I just tried it out, got the out of memory, and gave up on that approach. I guess I could bump it up to 1 gig or more, just to prove the theory but of course at some point I'm squeamish about wasting address space or actual memory. 1gig is tiny for address space but not for actual memory. (assuming a full 64 bit address space, which actually is probably not available by far) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 19:31:01 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 12:31:01 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: If sbrk works let's leave it at that for now. On 17 Nov 2008, at 07:00, Jay wrote: > > 1 meg pages didn't suffice > > Is this still the sidespan array problem? > > I assume so. > I just tried it out, got the out of memory, and gave up on that > approach. > I guess I could bump it up to 1 gig or more, just to prove the > theory but > of course at some point I'm squeamish about wasting address space or > actual memory. > 1gig is tiny for address space but not for actual memory. > (assuming a full 64 bit address space, which actually is probably > not available by far) > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Sun Nov 23 19:06:18 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 23 Nov 2008 12:06:18 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> References: <491CB84D.4000008@wichita.edu> <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Message-ID: <49299B9A.7050103@wichita.edu> Olaf Wagner wrote: > Hm, now I seem to be the victim of mail truncation. I'll try a top-quote > this time (our system administrator will investigate this). Third attempt: > > It seems that this time my email wasn't complete, for whatever reasons. > I was out of town for some days so I noticed only today. > > Rodney, I did ask you if you could write up some paragraphs for the > README in scripts and the web pages that clarifies this matter so that > others won't run into it again? Done. > > I always seem to miss some crucial points when trying to explain the > build system myself... > > Thanks in advance, > > Olaf > > Quoting "Rodney M. Bates" : > >> Here is how the undefined ROOT problem happens when trying to rebuild >> cm3 with itself. >> >> I had been making the assumption that commands build, ship, clean, when >> supplied to the do-cm3-*.sh scripts, did the same things as -build, >> -ship, -clean supplied to the cm3 command. Not so. >> >> build actually does cm3 -build -override, as do buildlocal and empty >> commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> There is a buildglobal command that does -build without -override, but >> it also does a ship as well. There appears to be no way to just build, >> without either overriding or shipping. >> >> So, I have been running do-cm3-*.sh with no argument (and sometimes build) From michael.anderson at elego.de Wed Nov 26 03:10:24 2008 From: michael.anderson at elego.de (Michael Anderson) Date: Wed, 26 Nov 2008 03:10:24 +0100 Subject: [M3devel] compiling cvsup on Debian Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Howdy there, I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using this thread as starting point: https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html Thus far, I've: - applied the patch - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;" - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", as well as the corresponding procedure calls as outlined here: https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html Which that leaves me with just one teensy error: "../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait) which does go away if I change the IMPORT and procedure calls back to SchedulerPosix for JUST SigHandler.m3. Is it safe to use both "Scheduler" and "SchedulerPosix", or will Bad_Things happen? Thanks, mike -- Michael Anderson IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 michael.anderson at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Nov 26 17:03:52 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 26 Nov 2008 10:03:52 -0600 Subject: [M3devel] compiling cvsup on Debian In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu> Should be just fine! On 25 Nov 2008, at 20:10, Michael Anderson wrote: > Howdy there, > > I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, > using this thread as > starting point: > https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html > > Thus far, I've: > > - applied the patch > - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF > Utime.struct_timezone;" > - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", > as well as the > corresponding procedure calls as outlined here: > > https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html > > Which that leaves me with just one teensy error: > > "../src/SigHandler.m3", line 221: unknown qualification > '.' (IOAlertWait) > > which does go away if I change the IMPORT and procedure calls back > to SchedulerPosix for > JUST SigHandler.m3. > > Is it safe to use both "Scheduler" and "SchedulerPosix", or will > Bad_Things happen? > > Thanks, > mike > > > -- > Michael Anderson > IT Services & Support > > elego Software Solutions GmbH > Gustav-Meyer-Allee 25 > Building 12.3 (BIG) room 227 > 13355 Berlin, Germany > > phone +49 30 23 45 86 96 michael.anderson at elegosoft.com > fax +49 30 23 45 86 95 http://www.elegosoft.com > > Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin > Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 > > From rodney.bates at wichita.edu Tue Nov 4 15:45:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 04 Nov 2008 08:45:35 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <4910600F.7000009@wichita.edu> I have new and greatly expanded documentation on m3gdb. It is created in docbook format, and I have managed to get it to convert to one-page html. I intend to do multi-page html and pdf also. I am not sure where in the repository this should go, where to put html files so they are available on the web, or what to do with mechanically derived files in the repository. I thought of creating a new subdirectory doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary file, not a directory. Also, I presume that Olaf will want to get the html to look like the other pages, which some kind of style sheet should do, but I don't know where to find that either. Any help will be appreciated. Rodney Bates From wagner at elegosoft.com Tue Nov 4 17:04:32 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 04 Nov 2008 17:04:32 +0100 Subject: [M3devel] New m3gdb documentation In-Reply-To: <4910600F.7000009@wichita.edu> References: <4910600F.7000009@wichita.edu> Message-ID: <20081104170432.1nn4yh39444sc48k@mail.elegosoft.com> Quoting "Rodney M. Bates" : > I have new and greatly expanded documentation on m3gdb. It is created > in docbook format, and I have managed to get it to convert to one-page > html. I intend to do multi-page html and pdf also. All documentation is checked-in below cm3/doc and cm3/www. m3gdb should probably be integrated into doc. > I am not sure where in the repository this should go, where to put html > files so they are available on the web, or what to do with mechanically > derived files in the repository. I thought of creating a new subdirectory > doc/help/m3gdb, but I can only find how to get cvs to add a new ordinary > file, not a directory. just use `cvs add ' > Also, I presume that Olaf will want to get the html to look like the other > pages, which some kind of style sheet should do, but I don't know where to > find that either. The style sheet is normal.css at the top level. You can reference it like this in the HTML header: It's checked in at cm3/www/normal.css. When you've checked-in all your additions, we need to ship them to the WWW server to make them visible. I can do that if you like, just let me know when you're finished. It's great that you're taking care of m3gdb and thus providing a working debugger for all M3 users! Regards, Olaf > Any help will be appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney.bates at wichita.edu Thu Nov 6 17:31:23 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 06 Nov 2008 10:31:23 -0600 Subject: [M3devel] New m3gdb documentation Message-ID: <49131BDB.20903@wichita.edu> There is a new and greatly expanded web page with documentation on m3gdb, the Modula-3 debugger, located at: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html It's also accessible with more indexing, through the normal Modula-3 web pages, for example, starting at: http://modula3.elegosoft.com/cm3 and clicking through: -> Documentation -> CM3 Operations Guide-> m3gdb Rodney M. Bates From jay.krell at cornell.edu Sat Nov 8 16:40:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:40:50 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Message-ID: Tony, how about switching these #if's? I explain some of why below. parse.c: static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The first code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } You should be able to repro this by configuring m3cg for mips64-openbsd. I've only tested on cygwin host, but I could check others.. Thanks, - Jay From jay.krell at cornell.edu Sat Nov 8 16:45:15 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 8 Nov 2008 15:45:15 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? Message-ID: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From hosking at cs.purdue.edu Sat Nov 8 23:55:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 8 Nov 2008 22:55:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 02:25:32 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 01:25:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: AMD64_LINUX_RTLinker.mc Type: application/octet-stream Size: 17622 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MIPS64_OPENBSD_RTLinker.mc Type: application/octet-stream Size: 17692 bytes Desc: not available URL: From hosking at cs.purdue.edu Sun Nov 9 11:01:58 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 10:01:58 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 12:10:54 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 11:10:54 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 9 15:09:10 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 9 Nov 2008 14:09:10 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen Message-ID: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: ****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Nov 9 19:02:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 9 Nov 2008 18:02:41 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: Message-ID: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 09:28:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 08:28:18 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END;TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()=VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END;END F1; BEGIN F1();END t. Darn. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 18:02:41 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails:****** runtime error:*** NEW() was unable to allocate more memory.*** file "../src/runtime/common/RTAllocator.m3", line 285***Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *)PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always.I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 10 12:13:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 10 Nov 2008 11:13:05 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> Message-ID: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: > @M3paranoidgc always succeeds. I see, reading the code, it only does > reads. Nevertheless. > > Simply programs that just allocate memory forever also succeed. > Such as this: > > jay at amd64a:/dev2/cm3/t$ cat src/t.m3 > MODULE t EXPORTS Main; > TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; > TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; > > VAR a: REFANY; > > PROCEDURE F1()= > VAR b: REFANY; > BEGIN > WHILE TRUE DO > b := NEW(REF T2); > a := NEW(REF T1); > END; > END F1; > > BEGIN > F1(); > END t. > > Darn. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 18:02:41 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] AMD64_LINUX gc/stubgen > > > Clearly a problem with GC. I'm guessing you have something wrong > (pointer alignment?) in the configuration. I suspect if you run > with @M3paranoidgc you'll trip over the problem. > > On 9 Nov 2008, at 14:09, Jay wrote: > > Tony, any tips on this one? :) > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB @M3nogc > > always succeeds. > > jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 - > sno RemoteView.T -T.M3IMPTAB > > usually fails: > > *** > *** runtime error: > *** NEW() was unable to allocate more memory. > *** file "../src/runtime/common/RTAllocator.m3", line 285 > *** > > Formatter.m3: > > CONST ChunkSize = 128; (* Must be a power of 2 *) > > PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = > BEGIN > WITH t = NEW(T) DO > t.wr := wr; > t.width := width; > t.indent := 0; > > t.numChars := 0; > > (* t.stream := RefStream{NIL, ..}; *) > > t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line > that fails > > Lowering chunk size to 64 seemed to make it succeed more often, but > not always. > I think I'll go back to MIPS64_OPENBSD..probably easier to track > that problem down. > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 10 20:35:19 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 10 Nov 2008 19:35:19 +0000 Subject: [M3devel] AMD64_LINUX gc/stubgen In-Reply-To: <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> References: <97806969-D1C7-4D77-8B54-466088B99937@cs.purdue.edu> <831F958E-6D14-4DDD-AEF1-7BF3AAA3BC1B@cs.purdue.edu> Message-ID: It /seems/ like any switch or switch combination makes it stop happening. Admittedly I haven't run a loop just a few runs by hand for each combo. Maybe have it record the allocation quantities and "play them back"? - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Mon, 10 Nov 2008 11:13:05 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Hmm. Can you try @M3paranoidgc @M3noincremental @M3nogenerational? On 10 Nov 2008, at 08:28, Jay wrote: @M3paranoidgc always succeeds. I see, reading the code, it only does reads. Nevertheless. Simply programs that just allocate memory forever also succeed. Such as this: jay at amd64a:/dev2/cm3/t$ cat src/t.m3 MODULE t EXPORTS Main; TYPE T1 = RECORD c : ARRAY [0..10000000] OF INTEGER; END; TYPE T2 = RECORD c : ARRAY [0..4095] OF INTEGER; END; VAR a: REFANY; PROCEDURE F1()= VAR b: REFANY; BEGIN WHILE TRUE DO b := NEW(REF T2); a := NEW(REF T1); END; END F1; BEGIN F1(); END t. Darn. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 18:02:41 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] AMD64_LINUX gc/stubgen Clearly a problem with GC. I'm guessing you have something wrong (pointer alignment?) in the configuration. I suspect if you run with @M3paranoidgc you'll trip over the problem. On 9 Nov 2008, at 14:09, Jay wrote: Tony, any tips on this one? :) jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB @M3nogc always succeeds. jay at amd64a:~/dev2/cm3/m3-ui/zeus/AMD64_LINUX$ /cm3/bin/stubgen -v1 -sno RemoteView.T -T.M3IMPTAB usually fails: *** *** runtime error: *** NEW() was unable to allocate more memory. *** file "../src/runtime/common/RTAllocator.m3", line 285 *** Formatter.m3: CONST ChunkSize = 128; (* Must be a power of 2 *) PROCEDURE New(wr: Wr.T; width: CARDINAL:= 75): T RAISES {} = BEGIN WITH t = NEW(T) DO t.wr := wr; t.width := width; t.indent := 0; t.numChars := 0; (* t.stream := RefStream{NIL, ..}; *) t.buffer := NEW (ExprBuf, 4 * ChunkSize); << this is the line that fails Lowering chunk size to 64 seemed to make it succeed more often, but not always. I think I'll go back to MIPS64_OPENBSD..probably easier to track that problem down. - Jay From rcoleburn at scires.com Tue Nov 11 00:17:31 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Mon, 10 Nov 2008 18:17:31 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081109062016.407EF10D5894@birch.elegosoft.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> Message-ID: <491879AF.1E75.00D7.1@scires.com> Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy >>> Jay Krell 11/9/2008 1:26 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/11/09 07:20:15 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files: cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message: begin combining nt386.common and unix.common (into cm3cg.common) mips64_openbsd support being migrating config-no-install/solgnu and solsun to Unix.common (had this sitting around, in intermediate state) go back to old formating because that is all the old C written m3build could handle add probing for host/target specific cm3cg (so I can stop copying it around as I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Nov 11 02:06:07 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 11 Nov 2008 01:06:07 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <491879AF.1E75.00D7.1@scires.com> References: <20081109062016.407EF10D5894@birch.elegosoft.com> <491879AF.1E75.00D7.1@scires.com> Message-ID: If you use my config files, yes. Note that there are two of them, one just includes the other. Take "the other". This is just because on my system, I think: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386 => \dev2\cm3.2\m3-sys\cminstall\src\config\NT386.common => \dev2\cm3.2\m3-sys\cminstall\src\config\cm3cfg.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common NT386 is the oddball for me, since for any other targets, for me, it goes: \cm3\cm3.cfg -- probes around => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\platform => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\Unix.common => \dev2\cm3.2\m3-sys\cminstall\src\config-no-install\cm3cfg.common I'd like to move NT386* to config-no-install, but I avoided doing so so far. I'd like to replace config with config-no-install, but avoid that too. - Jay Date: Mon, 10 Nov 2008 18:17:31 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Does this mean we will now need to put a "cm3cg.common" file into our bin folder on Windows? Just checking before I do any updates from the repository so I know what needs to change in my target . Regards, Randy>>> Jay Krell 11/9/2008 1:26 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/11/09 07:20:15Modified files:cm3/m3-sys/cminstall/src/config/: NT386.common cm3.cfg cm3/m3-sys/cminstall/src/config-no-install/: AMD64_LINUX LINUXLIBC6 PPC32_OPENBSD PPC_DARWIN PPC_LINUX SOLgnu SOLsun SPARC32_LINUX SPARC64_LINUX SPARC64_OPENBSD Unix.common Added files:cm3/m3-sys/cminstall/src/config-no-install/: MIPS64_OPENBSD cm3cfg.common Log message:begin combining nt386.common and unix.common (into cm3cg.common)mips64_openbsd supportbeing migrating config-no-install/solgnu and solsun to Unix.common(had this sitting around, in intermediate state)go back to old formating because that is all the old C written m3build could handleadd probing for host/target specific cm3cg (so I can stop copying it aroundas I switch targets) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Nov 11 21:39:09 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 11 Nov 2008 14:39:09 -0600 Subject: [M3devel] Still can't bootstrap cm3 Message-ID: <4919ED6D.6000700@wichita.edu> We still have serious problems with the cm3 system building. I have again gotten into a big hampster wheel of several different failures, and still can't rebuild a working compiler. I have gone through enough steps that I am not just positive I have remembered it all right, but it was something like this: I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and ./install-cm3-compiler.sh upgrade. That left me with a compiler that always failed with: ------------------------------------------------------------------------------------------------------------ rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake runtime error: undefined variable: ROOT --procedure-- -line- -file--- _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl _import_template 4 /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Then I tried rebuilding using various do-cm3-* scripts, including after supplying argument realclean (which, I have learned many times in the past, is essential to avoid all sorts of problems), with the same results. Somewhere in here, I started getting the bad version stamp failures too. Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, a package at a time, I was consistently getting: -------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build --- building in LINUXLIBC6 --- ignoring ../src/m3overrides "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime error: unable to open "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" for reading --procedure-- -line- -file--- import_version -- import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS include_dir 11 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile 4 /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args Fatal Error: package build failed ----------------------------------------------------------------------------------------------------------- Perhaps it was the realclean that deleted all the .M3EXPORTS files. ../boot_cm3-core.sh LINUXLIBC6 gave this failure: ----------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh please specify a cross compilation target platform rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh LINUXLIBC6 /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph parseparams realgeometry set slisp sortedtableextras table-list tempfiles === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ --- building in LINUXLIBC6 --- new source -> compiling RTHooks.i3 /bin/sh: /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not found m3_backend => 127 m3cc (aka cm3cg) failed compiling: RTHooks.ic --------------------------------------------------------------------------------------------------------- Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed on all the Modula-3 code). I got around the above by putting in a symbolic link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to cm3/m3-sys/m3cc/LINUXLIBC6, so it could find cm3cg. Then I went through the package-at-a-time order Tony had given in an old post to the m3devel list. That had other problems. Running cm3 in the package directory failed with the undefined ROOT problem, but do-pkg.sh would do the build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in each package with cm3 -ship. After building and shipping m3 middle and m3front, package cm3 itself wouldn't link, wanting libraries for m3back, m3objfile, m3quake, sysutils, m3linker, libm3, and m3core. I skipped it and m3gc*, which don't exist now, and went through the rest of the order, which completed without failures building. But that gave me a compiler that fails with a new problem I have never seen before: --------------------------------------------------------------------------------------------------------- rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./install-cm3-compiler.sh upgrade cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical *** *** runtime error: *** A compile-time type is missing. *** file "../src/Utils.m3" *** Aborted (core dumped) cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 /usr/local/cm3/bin/cm3- cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg /usr/local/cm3/bin/cm3cg- cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg ------------------------------------------------------------------------------------------------------------ Going back to an older cm3 makes that go away and makes the ROOT problem recur. Working around that with cm3 -DROOT="/home/rodney/proj/m3/cm3-new/cm3" get the compiler to run, but it doesn't do anything, because it thinks nothing needs to be recompiled. I could delete stuff, but I fear that would just start me around another iteration of the same thing. I don't believe I have ever gotten a bootstrap to work. I think I have always had to download a recent binary and go from there. Any help would be greatly appreciated. Rodney Bates From jay.krell at cornell.edu Wed Nov 12 05:04:44 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 04:04:44 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: Try starting from a recent snapshot, not necessarily a recent release, and running upgrade.sh or upgrade.py. ROOT should be set in the config file, or as you showed, on the command line. upgrade.sh/upgrade.py will figure it out and set it on the command line. I'd kind of like the compiler to set it itself, but that is kind of inappropriate. It's definitely possible that starting from too old of a release won't work. Avoid any script with "boot" in the name. Those are I believe for bringing up new targets and esp. cross building, not for updating mature working systems that can build natively. The dash in LINUXLIBC6-LINUXLIBC6 confirms it. The dash appears when the host and target are different. The first platform is the host, the second is the target. - Jay From wagner at elegosoft.com Wed Nov 12 08:18:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 12 Nov 2008 08:18:55 +0100 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <4919ED6D.6000700@wichita.edu> References: <4919ED6D.6000700@wichita.edu> Message-ID: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Quoting "Rodney M. Bates" : > We still have serious problems with the cm3 system building. > I have again gotten into a big hampster wheel of several different > failures, and still can't rebuild a working compiler. > I have gone through enough steps that I am not just positive > I have remembered it all right, but it was something like this: Rodney, it's not easy to understand what has gone wrong at this point. Usually, if you have a working system, cm3/scripts/upgrade.sh should be able to bootstrap to a new set of sources. This is tested every day in the tinderbox release tests, too. As the last builds seem to have succeeded, I don't think there is a real problem in our code base currently. You may want to have a look at test_build_current in cm3/scripts/regression/defs.sh to see what's done in the regression builds. I'd suggest you either start again from the last release or one of the latest snapshots available from the build machine. I assume you are not building a cross compiler, but just trying to perform an upgrade. Daily snapshots are available from http://www.opencm3.net/snaps/snapshot-index.html I hope this helps, Olaf > I started with ./do-cm3-front.sh, ./do-cm3-front.sh -ship, and > ./install-cm3-compiler.sh upgrade. That left me with a compiler > that always failed with: > > ------------------------------------------------------------------------------------------------------------ > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl", line 4: quake > runtime error: undefined variable: ROOT > > --procedure-- -line- -file--- > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > _import_template 4 > /usr/local/cm3/pkg/libm3/src/bundleintf/bundle-ov.tmpl > import 274 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Then I tried rebuilding using various do-cm3-* scripts, including after > supplying argument realclean (which, I have learned many times in the > past, is essential to avoid all sorts of problems), with the same results. > Somewhere in here, I started getting the bad version stamp failures too. > Eventually, trying to rebootstrap with ./upgrade.sh, and also manually, > a package at a time, I was consistently getting: > > -------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/m3-sys/m3middle$ cm3 -build > --- building in LINUXLIBC6 --- > > ignoring ../src/m3overrides > > "/usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS", line 4: quake runtime > error: unable to open > "/home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core/LINUXLIBC6/.M3EXPORTS" > for reading > > --procedure-- -line- -file--- > import_version -- > import 4 /usr/local/cm3/pkg/libm3/LINUXLIBC6/.M3EXPORTS > include_dir 11 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/src/m3makefile > 4 > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3middle/LINUXLIBC6/m3make.args > > Fatal Error: package build failed > > ----------------------------------------------------------------------------------------------------------- > > Perhaps it was the realclean that deleted all the .M3EXPORTS files. > ../boot_cm3-core.sh LINUXLIBC6 gave this failure: > > ----------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > please specify a cross compilation target platform > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ ./boot-cm3-core.sh > LINUXLIBC6 > /home/rodney/proj/m3/cm3-new/cm3/scripts/pkgmap.sh -c "cm3 -build > -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep" m3core libm3 > sysutils m3middle m3objfile m3linker m3back m3front m3quake cm3 > m3scanner m3tools m3cgcat m3cggen m3bundle bitvector digraph > parseparams realgeometry set slisp sortedtableextras table-list > tempfiles > === package /home/rodney/proj/m3/cm3-new/cm3/m3-libs/m3core === > +++ cm3 -build -override -DROOT='/home/rodney/proj/m3/cm3-new/cm3' > -DCM3_VERSION_TEXT='d5.7.0' -DCM3_VERSION_NUMBER='050700' > -DCM3_LAST_CHANGED='2008-03-16' -DM3_BOOTSTRAP=TRUE -keep +++ > --- building in LINUXLIBC6 --- > > new source -> compiling RTHooks.i3 > /bin/sh: > /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6/cm3cg: not > found > m3_backend => 127 > m3cc (aka cm3cg) failed compiling: RTHooks.ic > > --------------------------------------------------------------------------------------------------------- > > Fortunately, an earlier ./upgrade.sh had built a cm3cg (though it had failed > on all the Modula-3 code). I got around the above by putting in a symbolic > link from cm3/m3-sys/m3cc/LINUXLIBC6-LINUXLIBC6 to > cm3/m3-sys/m3cc/LINUXLIBC6, > so it could find cm3cg. > > Then I went through the package-at-a-time order Tony had given in an old post > to the m3devel list. That had other problems. Running cm3 in the package > directory failed with the undefined ROOT problem, but do-pkg.sh would do the > build. But do-pkg.sh doesn't appear to do a ship, so I had to do that in > each package with cm3 -ship. > > After building and shipping m3 middle and m3front, package cm3 > itself wouldn't > link, wanting libraries for m3back, m3objfile, m3quake, sysutils, > m3linker, libm3, > and m3core. I skipped it and m3gc*, which don't exist now, and went > through the > rest of the order, which completed without failures building. But that gave > me a compiler that fails with a new problem I have never seen before: > > --------------------------------------------------------------------------------------------------------- > > rodney at yellowstone:~/proj/m3/cm3-new/cm3/scripts$ > ./install-cm3-compiler.sh upgrade > cp_if: /usr/local/cm3/bin/cm3 and /usr/local/cm3/bin/cm3-d5.7.0 identical > cp_if: /usr/local/cm3/bin/cm3cg and /usr/local/cm3/bin/cm3cg-d5.7.0 identical > > > *** > *** runtime error: > *** A compile-time type is missing. > *** file "../src/Utils.m3" > *** > > Aborted (core dumped) > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/cm3/LINUXLIBC6/cm3 > /usr/local/cm3/bin/cm3- > cp /home/rodney/proj/m3/cm3-new/cm3/m3-sys/m3cc/LINUXLIBC6/cm3cg > /usr/local/cm3/bin/cm3cg- > cp /usr/local/cm3/bin/cm3- /usr/local/cm3/bin/cm3 > cp /usr/local/cm3/bin/cm3cg- /usr/local/cm3/bin/cm3cg > > ------------------------------------------------------------------------------------------------------------ > > Going back to an older cm3 makes that go away and makes the ROOT problem > recur. Working around that with cm3 > -DROOT="/home/rodney/proj/m3/cm3-new/cm3" > get the compiler to run, but it doesn't do anything, because it thinks > nothing needs to be recompiled. I could delete stuff, but I fear that would > just start me around another iteration of the same thing. > > I don't believe I have ever gotten a bootstrap to work. I think I > have always > had to download a recent binary and go from there. > > Any help would be greatly appreciated. > > Rodney Bates -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Nov 12 08:57:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 07:57:31 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Nov 12 09:05:03 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Nov 2008 08:05:03 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: ps: it takes over 20 seconds for MIPS64_OPENBSD to crash in RunMainBody, which makes me suspect the platform is impractically slow. I can only hope that optimization and/or not crashing makes it much faster. The bootstrap build -- assemble a bunch, compile a little C, link, is also slow though. System is 200MHz+ and 256meg RAM though, ought to suffice.. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Wed, 12 Nov 2008 07:57:31 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Nov 12 19:49:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 12 Nov 2008 13:49:19 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081112152142.02AEA10D5970@birch.elegosoft.com> References: <20081112152142.02AEA10D5970@birch.elegosoft.com> Message-ID: <491ADDBA.1E75.00D7.1@scires.com> Rodney: Is this change simply to make PM3 and CM3 more compatible wrt pickles, or does it alter current behavior for folks who only use CM3 ? (i.e., will it result in a change to the way CM3 pickles are represented that may affect current CM3 users?) Regards, Randy >>> Rodney M. Bates 11/12/2008 4:21 PM >>> CVSROOT:/usr/cvs Changes by:rodney at birch.08/11/12 16:21:41 Modified files: cm3/m3-libs/libm3/src/pickle/ver2/: Pickle2.m3 Log message: Pickles in both PM3 and CM3 have some hard-coded fingerprints for common types, but the values are not consistent about byte order. Fix CM2 pickle-reading code to accept values of types NULL, ROOT, UNTRACED ROOT, ADDRESS, AND REFANY, written by PM3 pickle-writing code. Probably, the latter two can't even happen in a pickle, unless somebody wrote a custom special for them. This still won't handle a more complex type that depends on these types, e.g., an object that contains a field of one of them. This latter is probably not feasable, because it would require "decrypting" fingerprints, and the hash algorithm for fingerprints is lossy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Nov 12 21:46:53 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 12 Nov 2008 20:46:53 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: Message-ID: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 01:04:32 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 00:04:32 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Nov 13 16:45:34 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 09:45:34 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> Message-ID: <491C4B9E.4010307@wichita.edu> Olaf Wagner wrote: > > it's not easy to understand what has gone wrong at this point. > Usually, if you have a working system, cm3/scripts/upgrade.sh > should be able to bootstrap to a new set of sources. This is tested > every day in the tinderbox release tests, too. As the last builds seem > to have succeeded, I don't think there is a real problem in our > code base currently. You may want to have a look at test_build_current > in cm3/scripts/regression/defs.sh to see what's done in the regression > builds. > > I'd suggest you either start again from the last release or > one of the latest snapshots available from the build machine. Yes, that's what I always end up doing. But it would be nice to be able someday to get my local compiler to rebuild itself :-(. But even that is not working for me. Reinstalling a snapshot from 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad version stamps on Compiler.Platform and Compiler.ThisPlatform, while compiling SocketPosix.m3. Going to the most recent (at the time) snapshot of 2008-11-12, do-cm3-core.sh will run without failures, but leaves a compiler that always gets the undefined ROOT failure, even trying to compile a trivial program. > I assume you are not building a cross compiler, but just trying > to perform an upgrade. That's right. The only reason I messed with the cross scripts was that I hoped they might take care of the necessary bootstrapping in a way the would work. > > Daily snapshots are available from > http://www.opencm3.net/snaps/snapshot-index.html > > I hope this helps, > > Olaf > > From jay.krell at cornell.edu Thu Nov 13 17:37:47 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:37:47 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Randy, ROOT not being defined is a function of your config files. Look at mine. Mine are willing to ferry it from an environment variable. The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required. ROOT is the root of the source tree, as you correctly?successfully worked around. You are supposed to be able to use the compiler without a source tree. But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself, but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he is saying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases. But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run, I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store, but I don't think it was worth the effort...the effort I put in to clone the .sh in .cmd. I think I punted doing that in the *.py, and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program. Hm. Could be your config file wants it? Could be my config files wants it? I guess the usual refrain -- send us your config file(s). I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Nov 13 17:54:55 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Nov 2008 16:54:55 +0000 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: Also, until you find something that works: export OMIT_GCC=yes Otherwise you are going to waste a lot more time experimenting. - Jay From: jay.krell at cornell.eduTo: rodney.bates at wichita.edu; m3devel at elegosoft.comDate: Thu, 13 Nov 2008 16:37:47 +0000Subject: Re: [M3devel] Still can't bootstrap cm3 Randy, ROOT not being defined is a function of your config files.Look at mine. Mine are willing to ferry it from an environment variable.The various scripts figure it out and set it on the command line. e.g. sysinfo.sh Granted..I don't think ROOT is supposed to be required.ROOT is the root of the source tree, as you correctly?successfully worked around.You are supposed to be able to use the compiler without a source tree.But you can't, um, build the source tree without a source tree. I am very tempted to have cm3.exe set ROOT itself,but it is a bit inappropriate actually. PERHAPS if it was renamed CM3_ROOT... Olaf isn't saying "just use a current release", he issaying "start with a current release" and then try to build locally. You cannot necessarily bootstrap from arbitrarily old releases.But you should definitely be able to bootstrap from current snapshots. It does work. Whenever you get into a "bad state" -- after any failed run,I encourage you to blow away the entire cm3 install and "start over". Several "partial builds" will leave you with a "mixed state". Some of the scripts do or used to "backup" the pkg store,but I don't think it was worth the effort...the effort I putin to clone the .sh in .cmd. I think I punted doing that in the *.py,and I don't miss it. I also recommend trying the *.py files. > that always gets the undefined ROOT failure, even trying to compile> a trivial program.Hm. Could be your config file wants it?Could be my config files wants it?I guess the usual refrain -- send us your config file(s).I can dig more later..maybe days..busy with other stuff.. - Jay> Date: Thu, 13 Nov 2008 09:45:34 -0600> From: rodney> To: m3devel> Subject: Re: [M3devel] Still can't bootstrap cm3> > Olaf wrote:> > > > it's not easy to understand what has gone wrong at this point.> > Usually, if you have a working system, cm3/scripts/upgrade.sh> > should be able to bootstrap to a new set of sources. This is tested> > every day in the tinderbox release tests, too. As the last builds seem> > to have succeeded, I don't think there is a real problem in our> > code base currently. You may want to have a look at test_build_current> > in cm3/scripts/regression/defs.sh to see what's done in the regression> > builds.> > > > I'd suggest you either start again from the last release or> > one of the latest snapshots available from the build machine.> > Yes, that's what I always end up doing. But it would be nice to be> able someday to get my local compiler to rebuild itself :-(.> > But even that is not working for me. Reinstalling a snapshot from> 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing> .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad> version stamps on Compiler.Platform and Compiler.ThisPlatform,> while compiling SocketPosix.m3.> > Going to the most recent (at the time) snapshot of 2008-11-12,> do-cm3-core.sh will run without failures, but leaves a compiler> that always gets the undefined ROOT failure, even trying to compile> a trivial program.> > > I assume you are not building a cross compiler, but just trying> > to perform an upgrade.> > That's right. The only reason I messed with the cross scripts was that> I hoped they might take care of the necessary bootstrapping in a way> the would work.> > > > > Daily snapshots are available from> > http://www.opencm3.net/snaps/snapshot-index.html> > > > I hope this helps,> > > > Olaf> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Nov 13 18:06:22 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 11:06:22 -0600 Subject: [M3devel] Still can't bootstrap cm3 In-Reply-To: <491C4B9E.4010307@wichita.edu> References: <4919ED6D.6000700@wichita.edu> <20081112081855.alr1p7k4ggw0k4k0@mail.elegosoft.com> <491C4B9E.4010307@wichita.edu> Message-ID: <77D02692-31FE-4326-8E0E-8758C323CDE9@cs.purdue.edu> This is odd. I've been rebuilding my local compiler by hand several times in the last few weeks. I will try the scripts and follow up. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 13 Nov 2008, at 09:45, Rodney M. Bates wrote: > Olaf Wagner wrote: >> it's not easy to understand what has gone wrong at this point. >> Usually, if you have a working system, cm3/scripts/upgrade.sh >> should be able to bootstrap to a new set of sources. This is tested >> every day in the tinderbox release tests, too. As the last builds >> seem >> to have succeeded, I don't think there is a real problem in our >> code base currently. You may want to have a look at >> test_build_current >> in cm3/scripts/regression/defs.sh to see what's done in the >> regression >> builds. >> I'd suggest you either start again from the last release or >> one of the latest snapshots available from the build machine. > > Yes, that's what I always end up doing. But it would be nice to be > able someday to get my local compiler to rebuild itself :-(. > > But even that is not working for me. Reinstalling a snapshot from > 2008-6-6 and a fresh cvs checkout, upgrade.sh gives the missing > .M3EXPORTS failure, and do-cm3-core.sh (after realclean) gets bad > version stamps on Compiler.Platform and Compiler.ThisPlatform, > while compiling SocketPosix.m3. > > Going to the most recent (at the time) snapshot of 2008-11-12, > do-cm3-core.sh will run without failures, but leaves a compiler > that always gets the undefined ROOT failure, even trying to compile > a trivial program. > >> I assume you are not building a cross compiler, but just trying >> to perform an upgrade. > > That's right. The only reason I messed with the cross scripts was > that > I hoped they might take care of the necessary bootstrapping in a way > the would work. > >> Daily snapshots are available from >> http://www.opencm3.net/snaps/snapshot-index.html >> I hope this helps, >> Olaf >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 00:29:17 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 17:29:17 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 Message-ID: <491CB84D.4000008@wichita.edu> Here is how the undefined ROOT problem happens when trying to rebuild cm3 with itself. I had been making the assumption that commands build, ship, clean, when supplied to the do-cm3-*.sh scripts, did the same things as -build, -ship, -clean supplied to the cm3 command. Not so. build actually does cm3 -build -override, as do buildlocal and empty commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. There is a buildglobal command that does -build without -override, but it also does a ship as well. There appears to be no way to just build, without either overriding or shipping. So, I have been running do-cm3-*.sh with no argument (and sometimes build). This starts a chain of events that eventually leads to an installed libm3 that always executes quake script bundle-ov.tmpl, and it requires that either M3BUNDLE or ROOT be defined, neither of which normally is. This will happen trying to compile any program that uses libm3, which is just about everything. We should document this somewhere. From jay.krell at cornell.edu Fri Nov 14 02:44:44 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 01:44:44 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: I don't know, but something doesn't seem right. D:\cm3>dir /s/b bund* *ov* *temp* *tmp*... D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = BIN_USE & SL & "m3bundle"end D:\dev2\cm3.2>dir /s/b /a-d bund*tmp*D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.htmlD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmplD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl :\dev2\cm3.2>dir /s/b /a-d bund*tmp*:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl bundle-ov.tmp: if defined("M3BUNDLE") _M3BUNDLE = M3BUNDLEelse _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & SL & "m3bundle", BUILD_DIR) M3BUNDLE = _M3BUNDLEend => this file requires ROOT or M3BUNDLE like you say, but this isn't normally used. D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintfD:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile if defined("_M3BUNDLE_OVERRIDE") template ("bundle-ov")endtemplate ("bundle") You defined _M3BUNDLE_OVERRIDE somewhere along the way? Try replacing the one file with the other.. - Jay> Date: Thu, 13 Nov 2008 17:29:17 -0600> From: rodney.bates at wichita.edu> To: m3devel at elegosoft.com> Subject: [M3devel] The undefined ROOT problem, rebuilding cm3> > Here is how the undefined ROOT problem happens when trying to rebuild> cm3 with itself.> > I had been making the assumption that commands build, ship, clean, when> supplied to the do-cm3-*.sh scripts, did the same things as -build,> -ship, -clean supplied to the cm3 command. Not so.> > build actually does cm3 -build -override, as do buildlocal and empty> commands to the scripts. clean does cm3 -clean -override, as does cleanlocal.> There is a buildglobal command that does -build without -override, but> it also does a ship as well. There appears to be no way to just build,> without either overriding or shipping.> > So, I have been running do-cm3-*.sh with no argument (and sometimes build).> This starts a chain of events that eventually leads to an installed libm3> that always executes quake script bundle-ov.tmpl, and it requires that> either M3BUNDLE or ROOT be defined, neither of which normally is. This> will happen trying to compile any program that uses libm3, which is just> about everything.> > We should document this somewhere. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 03:01:30 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 13 Nov 2008 20:01:30 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: References: <491CB84D.4000008@wichita.edu> Message-ID: <491CDBFA.7030500@wichita.edu> Jay wrote: > I don't know, but something doesn't seem right. > > D:\cm3>dir /s/b bund* *ov* *temp* *tmp* > ... > D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = BIN_USE & SL & "m3bundle" > end > > D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* > D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > :\dev2\cm3.2>dir /s/b /a-d bund*tmp* > :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl > > bundle-ov.tmp: > > if defined("M3BUNDLE") > _M3BUNDLE = M3BUNDLE > else > _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & "%s" & > SL & "m3bundle", BUILD_DIR) > M3BUNDLE = _M3BUNDLE > end > > => this file requires ROOT or M3BUNDLE like you say, but this isn't > normally used. > > D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf > D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile > > if defined("_M3BUNDLE_OVERRIDE") > template ("bundle-ov") > end > template ("bundle") > > You defined _M3BUNDLE_OVERRIDE somewhere along the way? Maybe I should have given more detail on the chain of events. You got most of it. The missing piece is: m3-libs/libm3/src/m3overrides contains: _M3BUNDLE_OVERRIDE = "T" So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the above file of quake code to be interpreted, defining _M3BUNDLE_OVERRIDE, and the rest is as you have noted. > > Try replacing the one file with the other.. > > - Jay > > > > > Date: Thu, 13 Nov 2008 17:29:17 -0600 > > From: rodney.bates at wichita.edu > > To: m3devel at elegosoft.com > > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 > > > > Here is how the undefined ROOT problem happens when trying to rebuild > > cm3 with itself. > > > > I had been making the assumption that commands build, ship, clean, when > > supplied to the do-cm3-*.sh scripts, did the same things as -build, > > -ship, -clean supplied to the cm3 command. Not so. > > > > build actually does cm3 -build -override, as do buildlocal and empty > > commands to the scripts. clean does cm3 -clean -override, as does > cleanlocal. > > There is a buildglobal command that does -build without -override, but > > it also does a ship as well. There appears to be no way to just build, > > without either overriding or shipping. > > > > So, I have been running do-cm3-*.sh with no argument (and sometimes > build). > > This starts a chain of events that eventually leads to an installed libm3 > > that always executes quake script bundle-ov.tmpl, and it requires that > > either M3BUNDLE or ROOT be defined, neither of which normally is. This > > will happen trying to compile any program that uses libm3, which is just > > about everything. > > > > We should document this somewhere. > From hosking at cs.purdue.edu Fri Nov 14 03:50:07 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 13 Nov 2008 20:50:07 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CDBFA.7030500@wichita.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> Message-ID: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> I generally run with buildship for these commands. On 13 Nov 2008, at 20:01, Rodney M. Bates wrote: > Jay wrote: >> I don't know, but something doesn't seem right. >> D:\cm3>dir /s/b bund* *ov* *temp* *tmp* >> ... >> D:\cm3\pkg\libm3\src\bundleintf\bundle.tmpl >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = BIN_USE & SL & "m3bundle" >> end >> D:\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> D:\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> :\dev2\cm3.2>dir /s/b /a-d bund*tmp* >> :\dev2\cm3.2\doc\help\gen_html\libm3\src\bundleintf\bundle.tmpl.html >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl >> :\dev2\cm3.2\m3-libs\libm3\src\bundleintf\bundle.tmpl >> bundle-ov.tmp: >> if defined("M3BUNDLE") >> _M3BUNDLE = M3BUNDLE >> else >> _M3BUNDLE = format(ROOT & SL & "m3-tools" & SL & "m3bundle" & SL & >> "%s" & >> SL & "m3bundle", BUILD_DIR) >> M3BUNDLE = _M3BUNDLE >> end >> => this file requires ROOT or M3BUNDLE like you say, but this isn't >> normally used. >> D:\dev2\cm3.2>dir /s/b m3makefile | findstr /i bundleintf >> D:\dev2\cm3.2\m3-libs\libm3\src\bundleintf\m3makefile >> if defined("_M3BUNDLE_OVERRIDE") >> template ("bundle-ov") >> end >> template ("bundle") >> You defined _M3BUNDLE_OVERRIDE somewhere along the way? > > Maybe I should have given more detail on the chain of events. > You got most of it. The missing piece is: > > m3-libs/libm3/src/m3overrides contains: > _M3BUNDLE_OVERRIDE = "T" > > So, 'do-cm3-*.sh build' runs 'cm3 -build -override', causing the > above file of quake code to be interpreted, defining > _M3BUNDLE_OVERRIDE, > and the rest is as you have noted. > >> Try replacing the one file with the other.. >> - Jay >> > Date: Thu, 13 Nov 2008 17:29:17 -0600 >> > From: rodney.bates at wichita.edu >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 >> > >> > Here is how the undefined ROOT problem happens when trying to >> rebuild >> > cm3 with itself. >> > >> > I had been making the assumption that commands build, ship, >> clean, when >> > supplied to the do-cm3-*.sh scripts, did the same things as -build, >> > -ship, -clean supplied to the cm3 command. Not so. >> > >> > build actually does cm3 -build -override, as do buildlocal and >> empty >> > commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> > There is a buildglobal command that does -build without - >> override, but >> > it also does a ship as well. There appears to be no way to just >> build, >> > without either overriding or shipping. >> > >> > So, I have been running do-cm3-*.sh with no argument (and >> sometimes build). >> > This starts a chain of events that eventually leads to an >> installed libm3 >> > that always executes quake script bundle-ov.tmpl, and it requires >> that >> > either M3BUNDLE or ROOT be defined, neither of which normally is. >> This >> > will happen trying to compile any program that uses libm3, which >> is just >> > about everything. >> > >> > We should document this somewhere. From wagner at elegosoft.com Fri Nov 14 09:35:07 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 14 Nov 2008 09:35:07 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081114093507.rp91pc17kg8cc8gw@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. Perhaps you could write up some paragraphs for the scripts' README and the web pages? I always seem to overlook or tend to obscure some crucial points for other users :-/ Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Fri Nov 14 12:08:17 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 11:08:17 +0000 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: > From: hosking> > I generally run with buildship for these commands. That, plus if you "buildlocal", you are supposed to be barred from "shipping". That is, if you build with -override, you aren't supposed to be able to "ship". "ship" is to copy from the source tree to the install tree. I think I did see a problem once where it let me ship "local" stuff. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Fri Nov 14 16:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Fri, 14 Nov 2008 09:26:05 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> References: <491CB84D.4000008@wichita.edu> <491CDBFA.7030500@wichita.edu> <74C195D7-3B89-4696-AB57-DE41DDD09E6D@cs.purdue.edu> Message-ID: <491D988D.9080804@wichita.edu> Tony Hosking wrote: > I generally run with buildship for these commands. And so will I, henceforth. Rodney Bates From jay.krell at cornell.edu Fri Nov 14 21:24:55 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 14 Nov 2008 20:24:55 +0000 Subject: [M3devel] Linux/amd64 Message-ID: I turned of gc just while running stubgen. Leading to: http://m3.elegosoft.com/cm3/uploaded-archives/cm3-std-POSIX-AMD64_LINUX-d5.7.0.tar.gz http://m3.elegosoft.com/cm3/uploaded-archives/cm3-min-POSIX-AMD64_LINUX-d5.7.0.tar.gz Please try them? See how well they work for you all? I haven't run most of the stuff, but cm3 does build itself. gc does usually?often work. (saying "posix" I feel is redundant...) - Jay From jay.krell at cornell.edu Sun Nov 16 16:31:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 15:31:25 +0000 Subject: [M3devel] duplicated code? Message-ID: Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and fails either there or soon thereafter. The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Nov 16 17:18:40 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Nov 2008 16:18:40 +0000 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: Er, the large allocation does come from the gc itself. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO I'll know more shortly. I guess..it looks like the heap can be discontiguous, but we do record keeping for what it all spans. The comments say: (* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff | p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5M GetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M GetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for. t1,t2,t3 are just the lengths of the strings being concatented. Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig. And it seems the heap allocator wants to allocate an array to describe the pages. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc. Perhaps perhaps blowing it up, to say, 1meg, will address this? But really, an array to describe pages spanning the results of separate memory allocations, seems wrong. A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code? Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 03:14:55 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 16 Nov 2008 20:14:55 -0600 Subject: [M3devel] duplicated code? In-Reply-To: References: Message-ID: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> On 16 Nov 2008, at 10:18, Jay wrote: > Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? No assumption. > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. Correct. > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. Ah, yes, that is most unfortunate. > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 09:39:48 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 08:39:48 +0000 Subject: [M3devel] AMD64_LINUX (again) Message-ID: ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8 ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Nov 17 10:10:43 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:10:43 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117101043.m08cyzrq8gkksco8@mail.elegosoft.com> Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Mon Nov 17 10:30:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 17 Nov 2008 10:30:00 +0100 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <491CB84D.4000008@wichita.edu> References: <491CB84D.4000008@wichita.edu> Message-ID: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Hm, now I seem to be the victim of mail truncation. I'll try a top-quote this time (our system administrator will investigate this). Third attempt: It seems that this time my email wasn't complete, for whatever reasons. I was out of town for some days so I noticed only today. Rodney, I did ask you if you could write up some paragraphs for the README in scripts and the web pages that clarifies this matter so that others won't run into it again? I always seem to miss some crucial points when trying to explain the build system myself... Thanks in advance, Olaf Quoting "Rodney M. Bates" : > Here is how the undefined ROOT problem happens when trying to rebuild > cm3 with itself. > > I had been making the assumption that commands build, ship, clean, when > supplied to the do-cm3-*.sh scripts, did the same things as -build, > -ship, -clean supplied to the cm3 command. Not so. > > build actually does cm3 -build -override, as do buildlocal and empty > commands to the scripts. clean does cm3 -clean -override, as does cleanlocal. > There is a buildglobal command that does -build without -override, but > it also does a ship as well. There appears to be no way to just build, > without either overriding or shipping. > > So, I have been running do-cm3-*.sh with no argument (and sometimes build). > This starts a chain of events that eventually leads to an installed libm3 > that always executes quake script bundle-ov.tmpl, and it requires that > either M3BUNDLE or ROOT be defined, neither of which normally is. This > will happen trying to compile any program that uses libm3, which is just > about everything. > > We should document this somewhere. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Nov 17 10:35:18 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 09:35:18 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: 1meg pages weren't large enough to avoid running out of memory. Any thoughts on using the boehm-gc? I don't know if it is compacting. Or even generational. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level). When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Either that, or a binary searched array where you only store entries for state changes. You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 13:21:26 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 06:21:26 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: On 17 Nov 2008, at 03:35, Jay wrote: > 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? > Any thoughts on using the boehm-gc? Hell no. > I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). > Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. > Or anything about it really, just that it is much ported and > presumably well maintained, since it is part of "gcc", part of the > gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. > Hm... I assume the right data structure here is a multi level array, > that you index by picking off progressively less significant bits of > an address (or page number) just like the hardware page tables for > virtual memory (but without the perf benefits of a cache and using > physical addresses after the first level). > When an entire level is at the same state, due to skipping a large > span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. > Either that, or a binary searched array where you only store entries > for state changes. > You'd use like the STL's upper_bound/lower_bound/equal_range > functions, that is, really the same thing as bsearch, but when an > entry isn't found, you return where it would be inserted, which is > like the last place you looked before giving up. I'm concerned about the lookup costs for this. > It might be worth confirming "that I'm not crazy" -- that mmap > behaves this way for other people, or that skimming its code > suggests it is not surprising. Could be related to how malloc works > also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". > > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 16 Nov 2008 20:14:55 -0600 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] duplicated code? > > > On 16 Nov 2008, at 10:18, Jay wrote: > > Er, the large allocation does come from the gc itself. > > What's being allocated? The side array should not be too huge. > > Tony, is there is an assumption that the heap is contiguous? > That calls to RTOS.GetMemory return adjacent addresses? > > No assumption. > > This code allocates a large array: > > IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN > WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO > > I'll know more shortly. > > I guess..it looks like the heap can be discontiguous, but > we do record keeping for what it all spans. > > Correct. > > The comments say: > (* The array desc and the global variables p0, and p1 describe the > pages > that are part of the traced heap. Either p0 and p1 are equal to > Nil and > no pages are allocated; or both are valid pages and page p is > allocated > iff > | p0 <= p < p1 > | AND desc[p - p0] != Unallocated > > > Hm.. > > Grow (0x44000) => 0x2b1e45256000 total: 1.5M > GetUntracedOpenArray(0x1a80) > span: 6.6M density: 24% > stubgen: Processing RemoteView.T > GetUntracedOpenArray(0x3f0) > t1:0xc > t2:0xa > t3:0x1 > Grow (0x52000) => 0x2aaaaaaab000 total: 1.8M > GetUntracedOpenArray(0x1ce69fc8) > > I have GetUntracedOpenArray printing how many bytes it is asked for. > t1,t2,t3 are just the lengths of the strings being concatented. > Grow(x)=>y means Grow allocated x bytes at address y. > > So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are > very far apart, like 400gig. > And it seems the heap allocator wants to allocate an array to > describe the pages. > > Ah, yes, that is most unfortunate. > > > Hm. Page size is no longer tied to the underlying system -- no > longer vm-tied gc. > Perhaps perhaps blowing it up, to say, 1meg, will address this? > > I am working on minimizing the need for the global array, but we do > need something that can be easily indexed like this. Perhaps we > need to pass a hint to RTOS.GetMemory that will try to allocate its > regions close together. > > But really, an array to describe pages spanning the results of > separate memory allocations, seems wrong. > A sparser data structure would be good, that could describe > arbitrary sized runs of pages as being in the same state. > > Indeed. As mentioned above, I am working to eliminate the need for > this. Code that starts us on this path will be checked in within a > day or so. > > - Jay > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sun, 16 Nov 2008 15:31:25 +0000 > Subject: [M3devel] duplicated code? > > Anyone want to clean up this duplicity? > > > D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3 > D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3 > D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3 > D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 > > > somewhere in there, AMD64_LINUX tries to allocate a lot of memory, > and fails > either there or soon thereafter. > The garbage collector is working. > > - Jay > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 13:48:59 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 12:48:59 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: understood, ok. I don't know the algorithm for allocating pages. There is a "hint" parameter to mmap, it isn't exactly clear how it is handled. Probably giving the block we have plus its size is reasonable. I can try that out at some point, though using sbrk seems ok. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: gc vs. large spans of memoryDate: Mon, 17 Nov 2008 06:21:26 -0600 On 17 Nov 2008, at 03:35, Jay wrote: 1meg pages weren't large enough to avoid running out of memory. Is this still the sidespan array problem? Any thoughts on using the boehm-gc? Hell no. I don't know if it is compacting. Indeed, Boehm's collector does not do compaction. Also, Boehm's is fully conservative ("ambiguous-roots + ambiguous heap"), though it is possible to configure it to use precise heap information (which CM3 does provide). Or even generational. It is generational, and has a parallel collection mode, but it is not concurrent like ours (we have a better chance of scaling on SMPs). Our collector will soon also be "on-the-fly", which means we won't need to have a stop-the-world phase where all the threads are stopped at the same time to initiate GC. Instead, we will simply signal threads one at a time to prepare for GC. Or anything about it really, just that it is much ported and presumably well maintained, since it is part of "gcc", part of the gcc Java support. Our collector is much ported and well-maintained too! ;-) Seriously, I am quite sure we have no collector bugs currently (it has been running reliably for many years now). The problems you are encountering are porting issues and not bugs in the collector. Also, the current collector is written in Modula-3, and nicely integrated with the Modula-3 object model and run-time. There is a *huge* amount to be said for eating your own dog-food by writing the collector in Modula-3. There is *no* good reason to step outside Modula-3 to C as would be needed for Boehm GC. Hm... I assume the right data structure here is a multi level array, that you index by picking off progressively less significant bits of an address (or page number) just like the hardware page tables for virtual memory (but without the perf benefits of a cache and using physical addresses after the first level).When an entire level is at the same state, due to skipping a large span, you just have one shared entry for all of them, major savings. Right. I have this partially implemented already. Either that, or a binary searched array where you only store entries for state changes.You'd use like the STL's upper_bound/lower_bound/equal_range functions, that is, really the same thing as bsearch, but when an entry isn't found, you return where it would be inserted, which is like the last place you looked before giving up. I'm concerned about the lookup costs for this. It might be worth confirming "that I'm not crazy" -- that mmap behaves this way for other people, or that skimming its code suggests it is not surprising. Could be related to how malloc works also. Are there hints that can be passed to mmap on that platform to cause less scattered mappings? It is odd that sbrk is less scattered, since it must also bottom out at mmap, but perhaps it is trying to maintain a reasonably compact allocation below the "brk". - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 16 Nov 2008 20:14:55 -0600CC: m3devel at elegosoft.comSubject: Re: [M3devel] duplicated code? On 16 Nov 2008, at 10:18, Jay wrote: Er, the large allocation does come from the gc itself. What's being allocated? The side array should not be too huge. Tony, is there is an assumption that the heap is contiguous?That calls to RTOS.GetMemory return adjacent addresses? No assumption. This code allocates a large array: IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DOI'll know more shortly. I guess..it looks like the heap can be discontiguous, butwe do record keeping for what it all spans. Correct. The comments say:(* The array desc and the global variables p0, and p1 describe the pages that are part of the traced heap. Either p0 and p1 are equal to Nil and no pages are allocated; or both are valid pages and page p is allocated iff| p0 <= p < p1| AND desc[p - p0] != Unallocated Hm.. Grow (0x44000) => 0x2b1e45256000 total: 1.5MGetUntracedOpenArray(0x1a80) span: 6.6M density: 24%stubgen: Processing RemoteView.TGetUntracedOpenArray(0x3f0)t1:0xct2:0xat3:0x1Grow (0x52000) => 0x2aaaaaaab000 total: 1.8MGetUntracedOpenArray(0x1ce69fc8) I have GetUntracedOpenArray printing how many bytes it is asked for.t1,t2,t3 are just the lengths of the strings being concatented.Grow(x)=>y means Grow allocated x bytes at address y. So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000 are very far apart, like 400gig.And it seems the heap allocator wants to allocate an array to describe the pages. Ah, yes, that is most unfortunate. Hm. Page size is no longer tied to the underlying system -- no longer vm-tied gc.Perhaps perhaps blowing it up, to say, 1meg, will address this? I am working on minimizing the need for the global array, but we do need something that can be easily indexed like this. Perhaps we need to pass a hint to RTOS.GetMemory that will try to allocate its regions close together. But really, an array to describe pages spanning the results of separate memory allocations, seems wrong.A sparser data structure would be good, that could describe arbitrary sized runs of pages as being in the same state. Indeed. As mentioned above, I am working to eliminate the need for this. Code that starts us on this path will be checked in within a day or so. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Sun, 16 Nov 2008 15:31:25 +0000Subject: [M3devel] duplicated code?Anyone want to clean up this duplicity? D:\dev2\cm3.2>dir /s/b asttotype.m3 StubCode.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3 somewhere in there, AMD64_LINUX tries to allocate a lot of memory, and failseither there or soon thereafter.The garbage collector is working. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Nov 17 14:00:01 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 17 Nov 2008 13:00:01 +0000 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: > 1 meg pages didn't suffice > Is this still the sidespan array problem? I assume so. I just tried it out, got the out of memory, and gave up on that approach. I guess I could bump it up to 1 gig or more, just to prove the theory but of course at some point I'm squeamish about wasting address space or actual memory. 1gig is tiny for address space but not for actual memory. (assuming a full 64 bit address space, which actually is probably not available by far) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Nov 17 19:31:01 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 17 Nov 2008 12:31:01 -0600 Subject: [M3devel] gc vs. large spans of memory In-Reply-To: References: <4B6BDC53-DB07-4498-A0D4-CB383DFFD487@cs.purdue.edu> Message-ID: If sbrk works let's leave it at that for now. On 17 Nov 2008, at 07:00, Jay wrote: > > 1 meg pages didn't suffice > > Is this still the sidespan array problem? > > I assume so. > I just tried it out, got the out of memory, and gave up on that > approach. > I guess I could bump it up to 1 gig or more, just to prove the > theory but > of course at some point I'm squeamish about wasting address space or > actual memory. > 1gig is tiny for address space but not for actual memory. > (assuming a full 64 bit address space, which actually is probably > not available by far) > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Sun Nov 23 19:06:18 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Sun, 23 Nov 2008 12:06:18 -0600 Subject: [M3devel] The undefined ROOT problem, rebuilding cm3 In-Reply-To: <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> References: <491CB84D.4000008@wichita.edu> <20081117103000.ld2m2zsu8w48o4ck@mail.elegosoft.com> Message-ID: <49299B9A.7050103@wichita.edu> Olaf Wagner wrote: > Hm, now I seem to be the victim of mail truncation. I'll try a top-quote > this time (our system administrator will investigate this). Third attempt: > > It seems that this time my email wasn't complete, for whatever reasons. > I was out of town for some days so I noticed only today. > > Rodney, I did ask you if you could write up some paragraphs for the > README in scripts and the web pages that clarifies this matter so that > others won't run into it again? Done. > > I always seem to miss some crucial points when trying to explain the > build system myself... > > Thanks in advance, > > Olaf > > Quoting "Rodney M. Bates" : > >> Here is how the undefined ROOT problem happens when trying to rebuild >> cm3 with itself. >> >> I had been making the assumption that commands build, ship, clean, when >> supplied to the do-cm3-*.sh scripts, did the same things as -build, >> -ship, -clean supplied to the cm3 command. Not so. >> >> build actually does cm3 -build -override, as do buildlocal and empty >> commands to the scripts. clean does cm3 -clean -override, as does >> cleanlocal. >> There is a buildglobal command that does -build without -override, but >> it also does a ship as well. There appears to be no way to just build, >> without either overriding or shipping. >> >> So, I have been running do-cm3-*.sh with no argument (and sometimes build) From michael.anderson at elego.de Wed Nov 26 03:10:24 2008 From: michael.anderson at elego.de (Michael Anderson) Date: Wed, 26 Nov 2008 03:10:24 +0100 Subject: [M3devel] compiling cvsup on Debian Message-ID: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Howdy there, I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, using this thread as starting point: https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html Thus far, I've: - applied the patch - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF Utime.struct_timezone;" - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", as well as the corresponding procedure calls as outlined here: https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html Which that leaves me with just one teensy error: "../src/SigHandler.m3", line 221: unknown qualification '.' (IOAlertWait) which does go away if I change the IMPORT and procedure calls back to SchedulerPosix for JUST SigHandler.m3. Is it safe to use both "Scheduler" and "SchedulerPosix", or will Bad_Things happen? Thanks, mike -- Michael Anderson IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 michael.anderson at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From hosking at cs.purdue.edu Wed Nov 26 17:03:52 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 26 Nov 2008 10:03:52 -0600 Subject: [M3devel] compiling cvsup on Debian In-Reply-To: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> References: <20081126031024.o8hbvsdkgo0kw884@mail.elegosoft.com> Message-ID: <192844D9-5F3B-4E15-B2E9-C540D3EBD15B@cs.purdue.edu> Should be just fine! On 25 Nov 2008, at 20:10, Michael Anderson wrote: > Howdy there, > > I'm trying to compile cvsup-snap-16.1h on Debian Etch with cm3, > using this thread as > starting point: > https://mail.elegosoft.com/pipermail/m3devel/2007-June/000468.html > > Thus far, I've: > > - applied the patch > - changed line 744 of FileAttr.m3 to "tz: UNTRACED REF > Utime.struct_timezone;" > - changed all IMPORT directives for "SchedulerPosix" to "Scheduler", > as well as the > corresponding procedure calls as outlined here: > > https://mail.elegosoft.com/pipermail/m3devel/2008-January/001210.html > > Which that leaves me with just one teensy error: > > "../src/SigHandler.m3", line 221: unknown qualification > '.' (IOAlertWait) > > which does go away if I change the IMPORT and procedure calls back > to SchedulerPosix for > JUST SigHandler.m3. > > Is it safe to use both "Scheduler" and "SchedulerPosix", or will > Bad_Things happen? > > Thanks, > mike > > > -- > Michael Anderson > IT Services & Support > > elego Software Solutions GmbH > Gustav-Meyer-Allee 25 > Building 12.3 (BIG) room 227 > 13355 Berlin, Germany > > phone +49 30 23 45 86 96 michael.anderson at elegosoft.com > fax +49 30 23 45 86 95 http://www.elegosoft.com > > Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin > Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 > >