From jay.krell at cornell.edu Sat Dec 13 19:03:08 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 13 Dec 2008 18:03:08 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a);END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:08:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:08:58 +0000 Subject: [M3devel] code duplication.. Message-ID: Anyone want to combine these identical files into one shared one somewhere? m3-mail/postcard/src/OSUtils.m3m3-mail/webcard/src/OSUtils.m3 - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:53:36 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:53:36 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 14 07:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 14 Dec 2008 17:23:06 +1100 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: > I tried getting C to error this way, and failed. > It looks like C uses COMPONENT_REF here, which will send it down > different paths I didn't follow. > COMPONENT_REF makes sense for it and only somewhat sense for us. > That would involve declaring all of a module's globals a struct, > which I don't believe we currently do but might be good. I think > there's a basic disconnect in that in C, bitfields are always > fields, never freestanding. Modula-3 rather violates that. Seems a > little dangerous therefore to be using them this way? > > In terms of the inefficient option in parse.c, that would maybe go > away, as all the offseting would be handled by the next level down, > like, implementation of fields? > Well..that is assuming the only use offset != 0 in m3_load is for > accessing module globals. I don't know if that is true. > > Maybe another option, should be easy to implement but might compile > slowly, is to cons up a field decl right there in m3_load/store? > That would lead to a lot of extra nodes -- per access instead of > declared once. I can try that out. > > - Jay > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sat, 13 Dec 2008 18:03:08 +0000 > > I've just spent a few hours debugging this. > > The problem is strongly related to the declaration that the global > segment is of size BIGGEST_ALIGNMENT. > gcc decides they can fit in a register..so then it doesn't bother > adding the offset to the address, something like that. > > When things work, it goes here: > op0 = adjust_address (op0, mode1, offset); > in expr.c. When things don't work, that line is skipped and then > shortly below that: > > if (mode1 != mode) > return convert_to_mode (tmode, op0, unsignedp); > return op0; << > > On systems where BIGGEST_ALIGNMENT fits in a register, e.g. > apparently MIPS, counter-e.g. apparently many others, but not > clearly all others. This determines whether the mode of the segment > is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger > than any machine type?). BLKmode works. It works on 32bit MIPS > because between mode1 and mode, one of them is SImode, one is > DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since > biggest alignment is set very large -- 128. I don't know why it > works for SPARC. > > PERHAPS I can put together a C or treelang example and get more (gcc > developers) eyes on it, but I doubt it. > > The following hack works around it: > > C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c > Index: parse.c > =================================================================== > RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v > retrieving revision 1.77 > diff -r1.77 parse.c > 2865c2865 > < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, > BIGGEST_ALIGNMENT); > --- > > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, > BIGGEST_ALIGNMENT); > > The repro is very easy. > Just add this to m3core: > > MODULE Main2 EXPORTS Main; > IMPORT RTIO; > VAR a := "a"; > BEGIN > RTIO.PutText(a); > END Main2. > > The load of a will be from MM_Main2 offset 0, instead of 104 or > thereabouts. > > I'll commit this shortly. > > - Jay > > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > Date: Thu, 13 Nov 2008 00:04:32 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > Jay, do you have access to an AMD64_DARWIN box? > > No, sorry, I don't yet and not sure when I will. > Unless it is easy enough to run on non-Apple hardware. > I'll have to debug it (m3cg) then. > > - Jay > > > > > > > > CC: m3devel at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Wed, 12 Nov 2008 20:46:53 +0000 > > Jay, do you have access to an AMD64_DARWIN box? I did get things > working on that 64-bit platform. Or is it an endianness issue you > are dealing with. In any case, I will not have a lot of bandwidth > for doing much in the way of tackling this unless it becomes > critical path for me soon. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 12 Nov 2008, at 07:57, Jay wrote: > > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:28:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:28:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct for size=align=maxalign, and then we access bit fields an offset from it. For if max is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:40:05 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:40:05 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: (truncated and reedited) Well, they are, definitely, but not with any fields, I think.Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it.If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang".Hopefully, going from just its name, it lets you build the exact trees that parse.c builds.As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supportedin current gcc without small patches (which cm3 applies).And then ask the gcc folks (cc you/all?)I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 10:20:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 09:20:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Treelang doesn't look hopeful. It is more like a "real language", though incomplete, and less about "direct tree building". I was hoping for something Lisp-like that mapped directly to the internal data structures. For now, I'm tired/lame/impatient, and will just stick with the "* 2" patch. Or maybe when we call relayout_decl, we can have a proper size. Maybe loads of module globals should go through some more specific than m3_load, and we can track the maximum offset. I'll try to write something reasonable to the gcc folks.. I could possibly write my own "treelang", but it doesn't seem worth the time currently, too much else to do. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: FW: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 14 Dec 2008 08:40:05 +0000 (truncated and reedited) Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it. If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 14 Dec 2008 17:23:06 +1100 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c Index: parse.c =================================================================== RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v retrieving revision 1.77 diff -r1.77 parse.c 2865c2865 < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT); --- > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main; IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a); END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu Date: Thu, 13 Nov 2008 00:04:32 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.com From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Wed, 12 Nov 2008 20:46:53 +0000 Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 10:01:58 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0 This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sat, 8 Nov 2008 22:55:54 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From jay.krell at cornell.edu Mon Dec 15 17:37:33 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 15 Dec 2008 16:37:33 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know. Not quite eveything is checked in and I didn't build a distribution yet. Within a few days, hopefully. I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 12:32:18 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 11:32:18 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass. Two are overly 32 bit specific. Two hang. Others fail that look familiar and/or floating point related, so probably fail on all. I didn't look too closely. The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap? I can try going back to that before debugging the hangs. In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited. Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 17:22:52 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 16:22:52 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I fixed the hang and updated the two archives. I had time_t wrong in some places. Are people on this list using any of: Linux/AMD64 FreeBSD/AMD64 NetBSD/AMD64 OpenBSD/AMD64 Solaris/AMD64 or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? or for that matter...? (I use Windows/AMD64 but am holding off on anything there.) - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Tue, 16 Dec 2008 11:32:18 +0000 please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass.Two are overly 32 bit specific.Two hang.Others fail that look familiar and/or floating point related, so probably fail on all.I didn't look too closely.The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs.In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited.Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Dec 16 20:33:27 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:33:27 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081215173318.0398710D5C74@birch.elegosoft.com> References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: This new file FilePosixC.c does not build on my SOLgnu Tinderbox setup, so the regressions failed last night. Jay, this is precisely the reason that we don't want to have C-coded files in the Modula-3 code-base -- C might proclaim its portability but that portability is a lie. Why do we need this when the old pure Modula-3 setup worked just fine! I know it meant some amount of duplication, but that duplication was needed because of differences in API specs on different platforms. I am not averse to having *portable* C code, but in this case I think it is too low-level to be properly portable. Perhaps better to keep it in Modula-3. On 15 Dec 2008, at 18:33, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/15 18:33:17 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > Log message: > Remove the need to untangle the #ifdefs around struct flock. From hosking at cs.purdue.edu Tue Dec 16 20:39:17 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:39:17 +1100 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: On 16 Dec 2008, at 22:32, Jay wrote: > please try out: > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > and/or > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > > Regression tests mostly pass. > Two are overly 32 bit specific. > Two hang. > Others fail that look familiar and/or floating point related, so > probably fail on all. > I didn't look too closely. > The hangs need investigation. > > Tony, should mmap work now, for arbitrarily distributed mmap? > I can try going back to that before debugging the hangs. Probably not. We still need a byte-sized entry per heap page that is maintained in the old array. I am working on developing a sparser representation for the 2 bits of information we still need to be able to extract given an arbitrary ambiguous root (i.e., something that looks like a pointer into the heap from the thread stacks). > > In one case I know the test didn't hang, but the test infra > structure hangs waiting for it. > > All relevant changes are commited. > Sorry this has taken so long.. > > - Jay > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Subject: RE: [M3devel] AMD64_LINUX (again) > Date: Mon, 15 Dec 2008 16:37:33 +0000 > > I never debugged the formsedit crash, but it seems to be gone now. > Possibly courtesy of pruning down the cloned headers, I don't know. > Not quite eveything is checked in and I didn't build a distribution > yet. > Within a few days, hopefully. > I need to try the regression tests too. > > - Jay > > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Mon, 17 Nov 2008 08:39:48 +0000 > Subject: [M3devel] AMD64_LINUX (again) > > ok, with the sbrk workaround now: > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > You don't have to build anything. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > You probably want to run ./do-cm3-std.py buildship. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz > You get to build your own of almost everything (./install- > config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py > buildship && ./install-front.py && ./do-cm3-std.py buildship) > > > OR you can bootstrap from an existing other system, roughly: > cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/ > python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py > AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target > machine and extract it, cd into it, make, and then you have native > cm3 and proceed to previous.. > > > I'm able to run Juno, Cube, Calculator, Mentor. > > Hm. formsedit crashes. Darn. > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 1140881760 (LWP 13270)] > 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Cannot > access memory at address 0x144006bf8 > ) > at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > 325 VBT.PaintTint(v, r, v.shadow.bg); > (gdb) c > Continuing. > Cannot find user-level thread for LWP 13242: generic error > (gdb) bt > #0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Ca > nnot access memory at address 0x144006bf8 > ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > Cannot fetch general-purpose registers for thread 1140881760: > generic error > (gdb) > > > still might be worth other people trying out.. > > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 01:32:06 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 00:32:06 +0000 Subject: [M3devel] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: Tony, Well, it is a matter of degree, but I really dislike the duplication and I believe the portability is plenty attainable. It also didn't compile for me on Cygwin, I did a bit more research and fixed that. It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't rebuild there yet. L_SET is not portable. - Jay> From: hosking at cs.purdue.edu> To: jkrell at elego.de> Date: Wed, 17 Dec 2008 06:33:27 +1100> CC: m3devel at elegosoft.com; m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3> > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > setup, so the regressions failed last night.> > Jay, this is precisely the reason that we don't want to have C-coded > files in the Modula-3 code-base -- C might proclaim its portability > but that portability is a lie. Why do we need this when the old pure > Modula-3 setup worked just fine! I know it meant some amount of > duplication, but that duplication was needed because of differences in > API specs on different platforms. I am not averse to having > *portable* C code, but in this case I think it is too low-level to be > properly portable. Perhaps better to keep it in Modula-3.> > On 15 Dec 2008, at 18:33, Jay Krell wrote:> > > CVSROOT: /usr/cvs> > Changes by: jkrell at birch. 08/12/15 18:33:17> >> > Modified files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3> > m3makefile> > Added files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c> >> > Log message:> > Remove the need to untangle the #ifdefs around struct flock.> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 01:49:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 11:49:36 +1100 Subject: [M3devel] [M3commit] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: I'll wait for tomorrow's regressions to come through and see if it is fixed now for my SOLgnu Tinderbox setup. I see the temptation to eliminate duplication, and for these sorts of POSIX APIs I would expect portable C code to work pretty well. On 17 Dec 2008, at 11:32, Jay wrote: > Tony, Well, it is a matter of degree, but I really dislike the > duplication > and I believe the portability is plenty attainable. It also didn't > compile > for me on Cygwin, I did a bit more research and fixed that. > It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't > rebuild there yet. > L_SET is not portable. > > - Jay > > > From: hosking at cs.purdue.edu > > To: jkrell at elego.de > > Date: Wed, 17 Dec 2008 06:33:27 +1100 > > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > > setup, so the regressions failed last night. > > > > Jay, this is precisely the reason that we don't want to have C-coded > > files in the Modula-3 code-base -- C might proclaim its portability > > but that portability is a lie. Why do we need this when the old pure > > Modula-3 setup worked just fine! I know it meant some amount of > > duplication, but that duplication was needed because of > differences in > > API specs on different platforms. I am not averse to having > > *portable* C code, but in this case I think it is too low-level to > be > > properly portable. Perhaps better to keep it in Modula-3. > > > > On 15 Dec 2008, at 18:33, Jay Krell wrote: > > > > > CVSROOT: /usr/cvs > > > Changes by: jkrell at birch. 08/12/15 18:33:17 > > > > > > Modified files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > > > m3makefile > > > Added files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > > > > > Log message: > > > Remove the need to untangle the #ifdefs around struct flock. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 02:00:35 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 01:00:35 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I asked mainly in case it was related to the hang, but it wasn't. Thanks, - Jay CC: m3devel at elegosoft.comFrom: hosking@ Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs. Probably not. ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 16 18:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 16 Dec 2008 11:26:05 -0600 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: <4947E4AD.8080705@wichita.edu> I am using Linux/AMD64. Jay wrote: > I fixed the hang and updated the two archives. > I had time_t wrong in some places. > > Are people on this list using any of: > Linux/AMD64 > FreeBSD/AMD64 > NetBSD/AMD64 > OpenBSD/AMD64 > Solaris/AMD64 > or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? > or for that matter...? > > (I use Windows/AMD64 but am holding off on anything there.) > > - Jay > From rcoleburn at scires.com Wed Dec 17 16:12:09 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 10:12:09 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> Message-ID: <4948D035.1E75.00D7.1@scires.com> Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 17:29:54 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 16:29:54 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4948D035.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:10:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:10:05 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:23:06 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: > Eh, not a big deal, you can cd to it and it just fine. > > I removed it from my distribution building and regular building. > It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering > wasn't working. > I "merely" need to fix one or the other or just comment it out when > building Cygwin, which is rare. > > See...I figured..that the Cygwin serial package should still use the > Win32 source. > However, that has a dependency on FileWin32.m3, which is only built > in "full Win32" platforms. > If you do try to compile FileWin32.m3, it errors, because it exports > File, and I guess maybe FileWin32. > Possibly I just need to have Quake do a bit of munging such that the > first line of FileWin32.m3 is altered. > The NT386 version would be: > MODULE FileWin32 EXPORTS File. > > The NT386GNU version would be: > MODULE FileWin32; > > something like that. > > Or push the contents into another module, and then have a thin NT386 > module export it as File. > > Anyway, it's not a huge deal. > The main downfall is I "temporarily" no longer build it, on any > platform. > > Fixing the filtering is a better "temporary" solution, since that'd > restore me to building it for all but NT386GNU. > > Another avenue is to see if the Posix version works on Cygwin. > I think the Cygwin serial users however are a "multiple small > intersection" -- people who use Modula-3 (small) intersected with > people who use Cygwin (medium) (probably at zero here) intersected > with people who use serial (very small) => very very very small, so > that's not worth getting it to work. > > - Jay > > > Date: Wed, 17 Dec 2008 10:12:09 -0500 > From: rcoleburn at scires.com > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > > Jay: > > Why are we skipping serial package? I use serial I/O on Windows. > If you need my sources, let me know and I'll supply. > > Regards, > Randy > > >>> Jay Krell 12/17/2008 3:47 PM >>> > CVSROOT:/usr/cvs > Changes by:jkrell at birch.08/12/17 15:47:34 > > Modified files: > cm3/scripts/python/: pylib.py make-dist.py > > Log message: > fix line endings and 'temporarily' skip serial package -- there's a > problem here on I386_CYGWIN I forgot about where essentially you > want to provide both FilePosix and FileWin32; FilePosix will export > File, FileWin32 will export FileWin32, so that portable Modula-3 can > use File/FilePosix and Win32 specific serial code can use FileWin32; > maybe some day we should try out this serial stuff, and /maybe/ the > Posix paths work on Cygwin? For now, just skip it. And fix line > endings. And try out the alternate 'clean' path in make-dist > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 22:36:36 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 21:36:36 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: I have a SOLsun/SOLgnu system at home. Wait a few more hours please? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:05:30 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:05:30 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: From: TonyTo: jay I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? It's more complicated than that. It isn't just one bit. There's a few factors, that people care more and less about. GUI library -- Win32 or X thread library -- pthreads or Win32 (or user threads) file system path representation -- forward slashes or backward slashes This seems to be what people care most about -- to see forward slashes. C compiler -- Visual C++ cl or Cygwin/GNU gcc linker -- Visual C++ link or Cygwin/GNU ld C runtime -- msvcr*.dll or cygwin1.dll integrated backend or not The integrated backend is way faster, but doesn't yet support 64bit LONGINT. which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/pdb Many of these factors are independent of each other, but not entirely. Compiler/linker directly lead to which debugger you can use, since they each use different formats. Well, you can use either debugger, but..without symbols..at least for some modules. The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBase, so you can't link to the Cygwin C runtime with theVisual C++ linker. This is probably fixable at least in one combination withan alias (ability to link Cygwin C runtime with Visual C++ linker; I don'tknow if Cygwin/GNU ld understands aliases). I have also found that Cygwin produces incorrectimport .libs, perhaps their linker somehow accepts them. Cygwin can program the Win32 GUI, but there is an issue with passing 8 byte structs by value to __stdcall functions, the function name isn't mangled correctly and linking fails. It is a bit of a combinatorial explosion.As the system stands, you can control each bit in the config file, andmultiple combinations might work, though I don't run any combinatorial tests. Three combinations are "identified" (named, spoken of) and have had some development testing.NT386, NT386MINGNU (which I think should be called I386_MINGWIN), and NT386GNU (which I think should be called I386_CYGWIN). Furthermore, you can imagine I386_INTERIX and I386_UWIN (I forget if that's the name).Interix and UWIN being two other very viable Posix layers on Windows.Interix has been renamed a few times. It is called "SFU" -- services for Unix, and I don't know,like SUA -- services for unix applications, and I think there's another name.It is a free download from Microsoft for x86 Windows < Vista, and is an optional component in Vista.It has its pluses and minuses..not now. Anyway, in conversation here, one of the main things that people DO NOT care aboutis which thread library is used. I believe it was asserted that PM3/NT386GNU did work, but that it might have used Win32 threads. And maybe never had any GUI? I tried getting pthreads to work, but gave up debugging it.Perhaps a fresh go at debugging it would pan out..but I'm really not too interested.. I also have no interest in user threads. I believe they are never warranted, excepton old systems that don't have kernel threads, like the hypothetical DJGPP port. I never brought up the issue of serial ports.I ASSUMED that almost nobody uses them, and anyone that does use them, doesn'tcare if they are implemented using Cygwin or Win32, as long as they work. I ASSUME there is no interop issue, like getting an int fd from the serial portand passing it to select() or close, or getting a Win32 HANDLE and passing itto WaitForSingleObject, but I could be wrong here, certainly. I didn't look into if Cygwin is complete enough to implement them.I consider serial ports to be kind of out on the fringe..that it wouldn't be surprising if Cygwin didn't implement them..but I didn't look. Heck, I tried building minicom on some non-Linux system(s) recently and it failed miserably. I don't know if there is even much Posix portability option for serial port programming. Sure..I debug over a serial port, and lately have machines I can't get videoout of (Itanium) or even seem to have no video hardware (Sun), so I use a serial consolefor initial install (recent achievement :) still haven't installed over network..), but that's still imho a big step away from writing code that uses the serial port.(And NetBSD doesn't support the local video on SGI O2, but I'm running OpenBSD there for now.) Anyway, there are a few ways to look at this issue.More and less trivial. A trivial view is: Get the filtering to work. Build serial on all platforms but Cygwin. Or see if the Posix code will work. I'd be surprised if the intersection of Cygwin x Modula-3 x serial port programmers is not 0, but could be. I actually don't think anyone is using Cygwin x Modula-3. Another view is: Get the Win32 serial code to work on Cygwin. The code is already there, and buildable, but building in a system that also builds FilePosix.m3 breaks Modula-3 rules, you end up with two implementations of the File interface. What you want, I think, is FilePosix.m3 to provide the File interface, and FileWin32.m3 to provide just its own specific FileWin32 interface. I think this is a simple transform of the code. Anyway, not a big deal...my tone here...is rushed, not rude, sorry.Maybe I'll get the truncation I deserve. :) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:07:33 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:07:33 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: <4949318C.1E75.00D7.1@scires.com> Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy >>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500 From: rcoleburn at scires.com To: jkrell at elego.de CC: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:24:38 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:24:38 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: <7D219C8E-022A-4196-8786-CA8F4278C010@cs.purdue.edu> Yeah, I would have thought of M3 on Cygwin as presenting a POSIX API (even if some system services like threads are implemented under the covers using Cygwin). To that extent, everything other than m3core, where threads live, should compile as a POSIX library. On 18 Dec 2008, at 09:07, Randy Coleburn wrote: > Jay, > > I agree with Tony. > > From my point of view, you are "breaking" the normal Windows install > in order to rectify a problem for Cygwin. I put "breaking" in > quotes because the install will appear to work, but the serial > package will be silently omitted. > > It may not be a "big deal" to you, but it presents another hurdle > for those who want to install on Windows, and this hurdle is > probably not documented yet. > > Regards, > Randy > > >>> Tony Hosking 12/17/2008 3:23 PM >>> > I would have thought of CygWin as a POSIX platform that just happens > to have Windows underneath. Did we go round this roundabout before? > > On 18 Dec 2008, at 03:29, Jay wrote: > >> Eh, not a big deal, you can cd to it and it just fine. >> >> I removed it from my distribution building and regular building. >> It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering >> wasn't working. >> I "merely" need to fix one or the other or just comment it out when >> building Cygwin, which is rare. >> >> See...I figured..that the Cygwin serial package should still use >> the Win32 source. >> However, that has a dependency on FileWin32.m3, which is only built >> in "full Win32" platforms. >> If you do try to compile FileWin32.m3, it errors, because it >> exports File, and I guess maybe FileWin32. >> Possibly I just need to have Quake do a bit of munging such that >> the first line of FileWin32.m3 is altered. >> The NT386 version would be: >> MODULE FileWin32 EXPORTS File. >> >> The NT386GNU version would be: >> MODULE FileWin32; >> >> something like that. >> >> Or push the contents into another module, and then have a thin >> NT386 module export it as File. >> >> Anyway, it's not a huge deal. >> The main downfall is I "temporarily" no longer build it, on any >> platform. >> >> Fixing the filtering is a better "temporary" solution, since that'd >> restore me to building it for all but NT386GNU. >> >> Another avenue is to see if the Posix version works on Cygwin. >> I think the Cygwin serial users however are a "multiple small >> intersection" -- people who use Modula-3 (small) intersected with >> people who use Cygwin (medium) (probably at zero here) intersected >> with people who use serial (very small) => very very very small, so >> that's not worth getting it to work. >> >> - Jay >> >> >> Date: Wed, 17 Dec 2008 10:12:09 -0500 >> From: rcoleburn at scires.com >> To: jkrell at elego.de >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] [M3commit] CVS Update: cm3 >> >> >> Jay: >> >> Why are we skipping serial package? I use serial I/O on Windows. >> If you need my sources, let me know and I'll supply. >> >> Regards, >> Randy >> >> >>> Jay Krell 12/17/2008 3:47 PM >>> >> CVSROOT:/usr/cvs >> Changes by:jkrell at birch.08/12/17 15:47:34 >> >> Modified files: >> cm3/scripts/python/: pylib.py make-dist.py >> >> Log message: >> fix line endings and 'temporarily' skip serial package -- there's a >> problem here on I386_CYGWIN I forgot about where essentially you >> want to provide both FilePosix and FileWin32; FilePosix will export >> File, FileWin32 will export FileWin32, so that portable Modula-3 >> can use File/FilePosix and Win32 specific serial code can use >> FileWin32; maybe some day we should try out this serial stuff, and / >> maybe/ the Posix paths work on Cygwin? For now, just skip it. And >> fix line endings. And try out the alternate 'clean' path in make-dist >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:27:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:27:36 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> Message-ID: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e- mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:30:07 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:30:07 -0500 Subject: [M3devel] serial2 package Message-ID: <494936D7.1E75.00D7.1@scires.com> Jay: I'm too busy right now to jump into the serial I/O stuff, so I am sending you a ZIP file containing my "serial2" package. This package is a replacement for the regular "serial" package. It has a WIN32 and a POSIX implementation, but the POSIX implementation was tailored for HP-UX and to my knowledge never tested on anything else. The WIN32 implementation works for me. It has some bug fixes and extras that the regular serial package does not and it has been tested on NT, 2000, and XP. You are welcome to use any part of this that is helpful. I know that emailing ZIP files is probably bad form these days, but you can scan it with your AV program. There are no binaries in here, just text files. Regards, Randy Coleburn -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: serial2.zip Type: application/x-zip-compressed Size: 24875 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Dec 17 23:32:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:32:53 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: I really don't think it's a big deal, to omit a minor package here or there. Imho people should be able to build whatever they need from source. Granted, I'm not where I want to be there yet, building the OS and such. I think most of the packages have no users at all. The web browsers in the tree for example seemed useless. I don't think Cygwin has any users either, so granted, this compromise was backwards. I could just comment it out for building Cygwin and not commit it. Anyway, I'll improve serial one way or the other. Like either just put it back, breaking Cygwin (if you use the .py files), or fix the filtering (could always filter in the m3makefile, easy). Randy, Tony's point I took to be different. His point is just let it build the Posix code. And indeed, I don't remember if I ever tried it. I just assumed serial programming to be a bit out on the fringe and not surprising if Cygwin doesn't support it, or even if it works on many Posix platforms at all. Serial is after all one of the packages with filtering "built in" just for it. I know minicom (just an example of a friendly serial app on GNU/Linux (e.g. Debian)) for example doesn't compile on some "Posix" platforms I have -- I forgot which I tried..could be any of AIX, Irix, OpenBSD... (was trying to bring up additional serial consoles besides a really noisy machine; the clear winner was Hyperterm on XP using a $20 USB to serial converter fyi...). - Jay Date: Wed, 17 Dec 2008 17:07:33 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy>>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building.It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working.I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source.However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms.If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32.Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered.The NT386 version would be:MODULE FileWin32 EXPORTS File. The NT386GNU version would be:MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal.The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin.I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:52:14 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:52:14 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some. Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable" You throw this word around with emphasis a lot. Portable Modula-3 code. The lie of portable C code. What do you mean? There are many ways to write portable and non-portable Modula-3 and C code. For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*. It's quite easy to do either way, in either language. I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile. They are both actually very portable, but less portable than calling libraries that wrap them. What I mean is, Posix open is implemented on tons of systems, but not all. Win32 CreateFile is also implemented on tons of systems, but not all. It depends partly how you quantify systems. Win32 Create 1) is implemented on over a billion actual machines. open is actually in msvcrt.dll also, but let's ignore it for this discussion. I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever. I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. (Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything. I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library. It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C. If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix. I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting. That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable. Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained. As well, there is OpenMP. Another angle actually you can play here is quality of compiler. I did some experimentation with a range of compilers I had..I guess I was using C++ at the time. Some older compilers really stunk. Like, with CFront, you couldn't have more than one return. Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation. Cheating I think. See also Python and Perl. Well, ok, Modula-3 wins here by virtue of having a good small language spec. You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Dec 18 00:13:12 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 23:13:12 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: ps: similar question: What does "low level" mean? hand.c and dtoa.c seem low level to me. I know, they are excused because they are rare and in m3core. FilePosixC.c does not. The stuff I did for stat, somewhat. The garbage collector does. But I can't define it, other than, uselessly "dealing with bits is low level", but int is_even(int a) { return !(a & 1); } doesn't seem very "low level", just slightly, and is very "portable". I was given a book about "complexity" as a gift. It makes a point that "complex" and "simple" are hard to define. Many things that seem the one, also seem the other. I have to say, I have whittled down unix/*.i3 a bunch, for all the "new" platforms, and I look at what is left, and I am inclined to keep "attacking" it, keep making it "smaller", like the bitflags in Ustat.i3. I am inclined to write something like this: Ustat.i3: TYPE StatFlags = RECORD X, R, W: INTEGER; (* maybe u_short *) END; PROCEDURE GetStatFlags(VAR StatFlags); UstatC.c typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; const static StatFlags_t StatsFlags = { X, R, W }; void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; Any module using any of the flags, would retrieve them in initialization. A bit inefficient. Portable. Cuts out many repeated lines. The C could would need a bit of #ifdefing, to handle platforms that don't have some of the flags, to make them 0. Again, very portable. Cuts out repetition. Is a little slower. I know it also might not work, like if there are switch statements or code needing actual constant constants. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] Fwd: Output from "cron" commandDate: Wed, 17 Dec 2008 22:52:14 +0000 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some.Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable"You throw this word around with emphasis a lot.Portable Modula-3 code.The lie of portable C code.What do you mean? There are many ways to write portable and non-portable Modula-3 and C code.For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*.It's quite easy to do either way, in either language.I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile.They are both actually very portable, but less portable than calling libraries that wrap them.What I mean is, Posix open is implemented on tons of systems, but not all.Win32 CreateFile is also implemented on tons of systems, but not all.It depends partly how you quantify systems.Win32 Create 1) is implemented on over a billion actual machines.open is actually in msvcrt.dll also, but let's ignore it for this discussion.I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever.I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad.(Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything.I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library.It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C.If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix.I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting.That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable.Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained.As well, there is OpenMP. Another angle actually you can play here is quality of compiler.I did some experimentation with a range of compilers I had..I guess I was using C++ at the time.Some older compilers really stunk. Like, with CFront, you couldn't have more than one return.Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation.Cheating I think. See also Python and Perl.Well, ok, Modula-3 wins here by virtue of having a good small language spec.You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Dec 18 00:38:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 10:38:06 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: <292A034D-ED30-407E-B773-F7DB513524EF@cs.purdue.edu> Think about the difference between safe and UNSAFE in Modula-3. On 18 Dec 2008, at 10:13, Jay wrote: > ps: similar question: > > What does "low level" mean? > > hand.c and dtoa.c seem low level to me. > I know, they are excused because they are rare and in m3core. > > FilePosixC.c does not. > The stuff I did for stat, somewhat. > > The garbage collector does. > But I can't define it, other than, uselessly "dealing with bits is > low level", but > int is_even(int a) { return !(a & 1); } doesn't seem very "low > level", just slightly, and is very "portable". > > I was given a book about "complexity" as a gift. > It makes a point that "complex" and "simple" are hard to define. > Many things that seem the one, also seem the other. > > I have to say, I have whittled down unix/*.i3 a bunch, for all the > "new" platforms, and I look at what is left, and I am inclined to > keep "attacking" it, keep making it "smaller", like the bitflags in > Ustat.i3. > > I am inclined to write something like this: > > Ustat.i3: > > TYPE StatFlags = RECORD > X, R, W: INTEGER; (* maybe u_short *) > END; > > PROCEDURE GetStatFlags(VAR StatFlags); > > UstatC.c > > typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; > const static StatFlags_t StatsFlags = { X, R, W }; > void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; > > Any module using any of the flags, would retrieve them in > initialization. > A bit inefficient. > Portable. > Cuts out many repeated lines. > > The C could would need a bit of #ifdefing, to handle platforms that > don't have some of the flags, to make them 0. > > Again, very portable. > Cuts out repetition. > Is a little slower. > I know it also might not work, like if there are switch statements > or code needing actual constant constants. > > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] Fwd: Output from "cron" command > Date: Wed, 17 Dec 2008 22:52:14 +0000 > > 1) I totally don't mind the emails. I can ignore them for a day or > so easily. :) > > > 2) C code in m3core or elsewhere. This is a philisophical question I > couldn't decide, interested in discussing some. > Originally I put "FilePosixC.c" in m3core, and called it from libm3. > But I think that's..not great. > > > 3) "portable" > You throw this word around with emphasis a lot. > Portable Modula-3 code. > The lie of portable C code. > What do you mean? > > > There are many ways to write portable and non-portable Modula-3 and > C code. > For example there were Modula-3 regression tests printing the size > of an integer. > Portable to lots of systems, but not all. > > I can write Modula-3 code or C code dependent or independent of > sizeof(int) or long or void*. > It's quite easy to do either way, in either language. > I don't think that proves anything. > > > Likewise, in both Modula-3 and C, I can easily directly call Posix > open, or Win32 CreateFile. > They are both actually very portable, but less portable than calling > libraries that wrap them. > What I mean is, Posix open is implemented on tons of systems, but > not all. > Win32 CreateFile is also implemented on tons of systems, but not all. > It depends partly how you quantify systems. > Win32 Create 1) is implemented on over a billion actual machines. > open is actually in msvcrt.dll also, but let's ignore it for this > discussion. > I've read that the "Unix workstation" market is only like in the > single digit millions, either sold per year or sold ever. > I realize those billion PCs that run Windows, can also run *BSD and > Linux, making this all gray. > > Now, another way to quantify this though, that I find "impressive", > is that Win32 CreateFile at least historically was implemented on > PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. > (Basically, NT has been ported among the most of any /commercial > closed source/ OS.) > > But again, I don't think this proves anything. > I can pretty equally easily write portable or non-portable C or > Modula, "at the library level". > > One angle you might argue, is that Modula-3 has a "built in" > portability layer/library. > It is, in a sense, easier to not call CreateFile or open in Modula-3 > than it is in C. > If that is what you mean, ok. > > HOWEVER, if you look at the Modula-3 system, the portability layer, > you know, it is forked in terms of Posix and Win32. If you limit > yourself to the Posix part of it, well, then, you can use Posix. > I realize Posix systems are not all equal, they are all supersets of > Posix, with some overlap in their supersetting. > That's why my checkin worked on some systems but not others. > > Also, with respect to I/O, if use just stdio -- very portable. > Threading -- finally, soon, multi threading will be very portable in > C++, at least to the implementations still being maintained. > As well, there is OpenMP. > > Another angle actually you can play here is quality of compiler. > I did some experimentation with a range of compilers I had..I guess > I was using C++ at the time. > Some older compilers really stunk. Like, with CFront, you couldn't > have more than one return. > Therefore, code like: > int even(int a) > { > if (a & 1) > return 0; > return 1; > } > > /could/ be deemed not portable. > > Modula-3 wins here by virtue of there being only one implementation. > Cheating I think. > > See also Python and Perl. > Well, ok, Modula-3 wins here by virtue of having a good small > language spec. > You know..Perl is speced as the language implemented by > perl.exe....so big and gnarly and impossible to predict..impossible > to write down...C++ is fairly well speced, but only through > mountains of paper and endless discussion and clarifications.... > > > - Jay > > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 09:27:36 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Fwd: Output from "cron" command > > > > No, broken briefly then fixed quickly is OK -- that's what > regressions are for. But on the other hand, testing before checkin > is probably nicer to the rest of the world. That way you don't get > whinging e-mails from me complaining that the Tinderbox regressions > broke. I suppose it depends which pain you prefer: my e-mails or > testing comprehensively. ;-) > > I would hope that these sorts of system-dependent changes are > minimized by keeping as much code in *portable* Modula-3 as > possible. To my mind, nothing outside of m3core should need to > escape to C. > > On 18 Dec 2008, at 09:24, Jay wrote: > > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 04:10:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:10:15 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." Message-ID: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- From mika at async.caltech.edu Thu Dec 18 04:33:46 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:33:46 -0800 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: Your message of "Wed, 17 Dec 2008 22:52:14 GMT." Message-ID: <200812180333.mBI3XktJ079016@camembert.async.caltech.edu> I don't know what kind of a programming Superman you are. I switched from C to Modula-3 in 1999 because I was tired of having a new crop of bugs every time my C programs touched a new OS. Or, on Linux, every time there was a new version of Linux out (i.e., every day, twice on Sundays?). Yes, they were in fact flaws in my code. I don't doubt that there could exist bug-free portable programs coded in C that do what my programs were supposed to do. But in my experience, very very few programmers know C well enough to write truly portable programs. I'm also not one of them even though I've written more C than I care to remember. In Modula-3, it just seems natural...in fact it takes effort to write a program that *isn't* portable, the opposite of C. In fact, when I was teaching CS2 at Caltech, our CS1-2-3 sequence was sometimes known as the "Useless Languages Sequence" (it was supposed to be intro to programming, program construction/basic algorithms and data structures, and software engineering). CS1 was MIT's 6.001 (in Scheme); CS2 I taught in Modula-3; and someone taught CS3 in Eiffel. Each of the years I taught CS2 I had some students babbling on about how they wanted to study the material in C or C++, because they knew it and thought the other students would be more employable (it would be good for them). Well these self-professed C experts as good as never could answer some basic questions about C that have some very odd answers... For instance, if the following is a complete C program... int main() { int a[10]; int *b; b=... } how many values can b (legally) have? I believe the answer is... 13? Don Knuth once sent me a check for $2.56 because I spotted some non-conforming C in Vol. 2... If you don't know all those rules and think about them CONSTANTLY you will one day run into a compiler that makes you a very unhappy programmer. Sigh. I remember debugging a program that was comparing function pointers for ordering once... on a Convex. Yes the program was incorrect---unlike any other types of pointers, function pointers can only be compared for equality---so you might say we got what we deserved? (Of course the bloody thing did precisely what the programmer wanted on SunOS/sparc, NetBSD/i386, OSF/1 on the alpha, NetBSD on the hp300, solaris/sparc and every other OS we ever tried it on; and it didn't crash immediately nor emit warnings on the Convex but crashed sometimes minutes (millions of instructions) later.) Well @#%@#%%@ like that just doesn't happen in Modula-3. Mika Jay writes: >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :) >=20 >=20 >2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some. >Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great. >=20 >=20 >3) "portable" >You throw this word around with emphasis a lot. >Portable Modula-3 code. >The lie of portable C code. >What do you mean? >=20 >=20 >There are many ways to write portable and non-portable Modula-3 and C code. >For example there were Modula-3 regression tests printing the size of an in= >teger. > Portable to lots of systems=2C but not all. >=20 >I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*. >It's quite easy to do either way=2C in either language. >I don't think that proves anything. >=20 >=20 >Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile. >They are both actually very portable=2C but less portable than calling libr= >aries that wrap them. >What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll. >Win32 CreateFile is also implemented on tons of systems=2C but not all. >It depends partly how you quantify systems. >Win32 Create 1) is implemented on over a billion actual machines. >open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion. >I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever. >I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray. >=20 >Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently x86=2C IA64=2C AMD64. Not bad. >(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.) >=20 >But again=2C I don't think this proves anything. >I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level". >=20 >One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library. >It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C. >If that is what you mean=2C ok. >=20 >HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix. >I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting. >That's why my checkin worked on some systems but not others. >=20 >Also=2C with respect to I/O=2C if use just stdio -- very portable. >Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained. >As well=2C there is OpenMP. >=20 >Another angle actually you can play here is quality of compiler. >I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time. >Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return. >Therefore=2C code like: > int even(int a) =20 > { if (a & 1) return 0=3B =20 > return 1=3B =20 > } >=20 >/could/ be deemed not portable. >=20 >Modula-3 wins here by virtue of there being only one implementation. >Cheating I think. >=20 >See also Python and Perl. >Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec. >You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications.... >=20 >=20 > - Jay > > > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output= > from "cron" command > > >No=2C broken briefly then fixed quickly is OK -- that's what regressions ar= >e for. But on the other hand=2C testing before checkin is probably nicer t= >o the rest of the world. That way you don't get whinging e-mails from me c= >omplaining that the Tinderbox regressions broke. I suppose it depends whic= >h pain you prefer: my e-mails or testing comprehensively. =3B-) > >I would hope that these sorts of system-dependent changes are minimized by = >keeping as much code in *portable* Modula-3 as possible. To my mind=2C not= >hing outside of m3core should need to escape to C. > >On 18 Dec 2008=2C at 09:24=2C Jay wrote: > >(critical typo -- you can provide a time for it to power back on.)I'm loath= >e to leave all my machines on and burn the electricity..I don't trust them = >to have good power management=2C lower power when idle.I should automate so= >mething here though=2C see if I can run Tinderbox=2C and see if I can get c= >ron to power on/off.Every time I have looked at the Tinderbox it seemed too= > difficult to run. On the other hand=2C I don't know if the status quo is s= >o bad.You tell me it's broken. I understand that is not ideal=2C and I shou= >ld fix it fairly asap=2C but is it terrible every so often?I do tend to at = >least build multiple platforms=2C even if not "all"..so not every checkin b= >reaks anything/everything. I figure LINUXLIBC6 is the most popular=2C and I= > can always test that on birch. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] = >Fwd: Output from "cron" commandDate: Thu=2C 18 Dec 2008 09:12:52 +1100 > > >My machine is in a machine room at Purdue=2C on all the time. > >On 18 Dec 2008=2C at 08:37=2C Jay wrote: > >I keep most of my machines powered off most of the time. Do you have automa= >tion that both runs a daily Tinderbox=2C and turns the machine on/off?My SG= >I machine at least has a nice feature where you can software power it off= >=2C and provide a time that it should power down. I haven't automated=2C bu= >t it seems ideal for daily Tinderboxes. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output fro= >m "cron" command > > >Jay=2C > >I am still getting a compile error on whatever changes you made... > >-- Tony > >Begin forwarded message: > > >From: Tony Hosking >Date: 18 December 2008 00:04:32 GMT+11:00 >To: hosking at cs.purdue.edu >Subject: Output from "cron" command > >Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.s= >hproduced the following output:TESTHOSTNAME=3DniagaraWS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=3D5.4.0INSTROOT_REL=3D/homes/h= >osking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm= >3-inst/niagara/prev-okINSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/l= >ast-okINSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE= >=3DPOSIXCM3_TARGET=3DSOLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-S= >OLgnu-5.4.0.tgzCM3CVSSERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegoso= >ft.com:/usr/cvsBINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.eleg= >osoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Bu= >ildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file = >/tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout=2C compile and tes= >t of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 200= >8.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return v= >alue: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN =3D 0--2008.12.17 = >06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile = >return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN =3D 1*** COM= >PILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cle= >aning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regress= >ion test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3te= >st log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hoskin= >g/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.= >stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-= >min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLg= >nu-*.htmlTESTHOSTNAME=3DniagaraWS=3D/homes/hosking/work/cm3-ws/niagara-2008= >-12-17-11-57-02LASTREL=3D5.4.0INSTROOT_REL=3D/homes/hosking/work/cm3-inst/n= >iagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok= >INSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=3D/= >homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=3DPOSIXCM3_TARGET=3DS= >OLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSS= >ERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvsBINDIST= >MIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=3D/homes/hosking/work/c= >m3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.ele= >gosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu = >SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-0= >65704-RGaGCn/log.txt---checkout=2C compile and test of cm3 ...2008.12.17 06= >:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/= >build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008= >.12.17 07:16:18]CHECKOUT_RETURN =3D 0--2008.12.17 07:16:21 -- compile in pr= >ogress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compi= >le 2008.12.17 08:01:31]COMPILE_RETURN =3D 02008.12.17 08:01:38 -- tests in = >progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-06= >5704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN =3D 02008.= >12.17 08:01:38 -- checkout=2C compile and test run done.---removing build t= >ree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../hom= >es/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes= >/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hoski= >ng/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*= >.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snap= >shot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcle= >aning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone.= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :)
> =3B
> =3B
>2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some.
>Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great.
> =3B
> =3B
>3) "portable"
>You throw this word around with emphasis a lot.
>Portable Modula-3 code.
>The lie of portable C code.
>What do you mean?
> =3B
> =3B
>There are many ways to write portable and non-portable Modula-3 and C code.= >
>For example there were Modula-3 regression tests printing the size of an in= >teger.
> =3B Portable to lots of systems=2C but not all.
> =3B
>I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*.
>It's quite easy to do either way=2C in either language.
>I don't think that proves anything.
> =3B
> =3B
>Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile.
>They are both actually very portable=2C but less portable than calling libr= >aries that wrap them.
>What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll.
>Win32 CreateFile is also implemented on tons of systems=2C but not all.
>It depends partly how you quantify systems.
>Win32 Create 1) is implemented on over a billion actual machines.
>open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion.
>I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever.
>I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray.
> =3B
>Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently =3Bx86=2C IA64=2C AMD64. Not bad.
>(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.)
> =3B
>But again=2C I don't think this proves anything.
>I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level".
> =3B
>One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library.
>It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C.
>If that is what you mean=2C ok.
> =3B
>HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix.
>I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting.
>That's why my checkin worked on some systems but not others.
> =3B
>Also=2C with respect to I/O=2C if use just stdio -- very portable.
>Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained.
>As well=2C there is OpenMP.
> =3B
>Another angle actually you can play here is quality of compiler.
>I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time.
>Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return.
>Therefore=2C code like:
> =3B int even(int a) =3B
> =3B { =3B
 =3B  =3B if (a &=3B 1) =3B
 = >=3B  =3B =3B =3B return 0=3B =3B
> =3B  =3B return 1=3B =3B
> =3B }
> =3B
>/could/ be deemed not portable.
> =3B
>Modula-3 wins here by virtue of there being only one implementation.
>Cheating I think.
> =3B
>See also Python and Perl.
>Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec.
>You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications....
> =3B
> =3B
> =3B- Jay


> >
>
>From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Thu=2C 18= > Dec 2008 09:27:36 +1100
CC: m3devel at elegosoft.com
Subject: Re: [M3de= >vel] Fwd: Output from "cron" command


>
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >
No=2C broken briefly then fixed quickly is OK -- that's what regressio= >ns are for.  =3BBut on the other hand=2C testing before checkin is prob= >ably nicer to the rest of the world.  =3BThat way you don't get whingin= >g e-mails from me complaining that the Tinderbox regressions broke.  = >=3BI suppose it depends which pain you prefer: my e-mails or testing compre= >hensively.  =3B=3B-)
>

>
I would hope that these sorts of system-dependent changes are minimize= >d by keeping as much code in *portable* Modula-3 as possible.  =3BTo my= > mind=2C nothing outside of m3core should need to escape to C.
=

>
>
On 18 Dec 2008=2C at 09:24=2C Jay wrote:

erchange-newline> >
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>(critical typo -- you can provide a time for it to power back on.)
I'm = >loathe to leave all my machines on and burn the electricity..
I don't tr= >ust them to have good power management=2C lower power when idle.
I shoul= >d automate something here though=2C see if I can run Tinderbox=2C and see i= >f I can get cron to power on/off.
Every time I have looked at the Tinder= >box it seemed too difficult to run.
 =3B
On the other hand=2C I d= >on't know if the status quo is so bad.
You tell me it's broken. I unders= >tand that is not ideal=2C and =3BI should fix it fairly asap=2C but is = >it terrible every so often?
I do tend to at least build multiple platfor= >ms=2C even if not "all"..so not every checkin breaks anything/everything.R> =3B
I figure LINUXLIBC6 is the most popular=2C and I can always t= >est that on birch.
 =3B
 =3B- Jay


>
>
From: =3Bailto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:EC_Apple-converted-space> =3B.edu">jay.krell at cornell.edu
Subject: Re: [M3devel] Fwd: Output from = >"cron" command
Date: Thu=2C 18 Dec 2008 09:12:52 +1100


>
T: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-= >INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLL= >APSE: separate"> >
tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica= >=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WH= >ITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"><= >SPAN class=3DEC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12p= >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate">-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb= >(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: n= >ormal=3B BORDER-COLLAPSE: separate">yle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B= > COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER= >-SPACING: normal=3B BORDER-COLLAPSE: separate">yle-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFO= >RM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: norma= >l=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >
My machine is in a machine room at Purdue=2C on all the time.
PAN>
> >
>
On 18 Dec 2008=2C at 08:37=2C Jay wrote:

interchange-newline> >
=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate"> >
na">I keep most of my machines powered off most of the time.
 =3B>Do you have automation that both runs a daily Tinderbox=2C and turns the m= >achine on/off?
My SGI machine at least has a nice feature where you can = >software power it off=2C and provide a time that it should power down. I ha= >ven't automated=2C but it seems ideal for daily Tinderboxes.
 =3B> =3B- Jay


>
>From: =3Bilto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:C_EC_Apple-converted-space> =3Bll.edu">jay.krell at cornell.edu
Date: Thu=2C 18 Dec 2008 07:10:05 +110= >0
CC: =3B"mailto:m3devel at elegosoft.com">m3devel at elegosoft.com
Subject: [M3dev= >el] Fwd: Output from "cron" command


>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate"> >
n style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: non= >e=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LE= >TTER-SPACING: normal=3B BORDER-COLLAPSE: separate">pple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-= >TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE= >: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">s=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helv= >etica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate">FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate">ACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0= >=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: nor= >mal=3B BORDER-COLLAPSE: separate">tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">ple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-T= >RANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE:= > normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helve= >tica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate"> >
Jay=2C
>

>
I am still getting a compile error on whatever changes you made...V> >

>
-- Tony
<= >/DIV>
>

>
Begin forwarded message:

ewline> >
>
>
= >ica color=3D#000000 size=3D3>From:ace> =3Belvetica size=3D3>Tony Hosking <=3Bu">hosking at cs.purdue.edu>=3B
>
= >ica color=3D#000000 size=3D3>Date:ace> =3Belvetica size=3D3>18 December 2008 00:04:32 GMT+11:00
>
= >ica color=3D#000000 size=3D3>To:e> =3Bvetica size=3D3>hosking at cs.purdue.= >edu
>
= >ica color=3D#000000 size=3D3>Subject:-space> =3B=3DHelvetica size=3D3>Output from "cron" command
>
MARGIN-RIGHT: 0px">
>
Your "cron" job on niagara.cs.purdue.edu
$HOME/cm3/scripts/regressi= >on/cron.sh

produced the following output:

TESTHOSTNAME=3Dniag= >ara
WS=3D/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04
LASTR= >EL=3D5.4.0
INSTROOT_REL=3D/homes/hosking/work/cm3-inst/niagara/rel-5.4.0= >
INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok
INSTROOT= >_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-ok
INSTROOT_CUR=3D/home= >s/hosking/work/cm3-inst/niagara/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET= >=3DSOLgnu
BINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tg= >z
CM3CVSSERVER=3Dbirch.elegosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com= >:/usr/cvs
BINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.= >elegosoft.com..
ssh birch.elegosoft.com ok
Building cm3.
Tinderbox= > Tree:  =3B =3B"cm3"
Buildname:  =3B =3B =3B =3B= > =3B =3B =3B"SOLgnu SunOS 5.10 niagara release-build"

cr= >eating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt

--->
checkout=2C compile and test of cm3 ...
2008.12.17 06:30:07 -- chec= >kout in progress.
[start checkout 2008.12.17 06:30:11]
cd /tmp/build-= >cm3-20081217-063006-fOaGeh/build
cvs return value: 0
[end checkout 20= 08.12.17 06:49:43]
CHECKOUT_RETURN =3D 0
--
2008.12.17 06:49:45 --= > compile in progress.
[start compile 2008.12.17 06:49:45]
compile ret= >urn value: 0
[end compile 2008.12.17 06:55:03]
COMPILE_RETURN =3D 1R>*** COMPILE FAILED
removing build tree /tmp/build-cm3-20081217-063006-= >fOaGeh ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/niag= >ara-*

cleaning regression test log files...
/homes/hosking/tmp/cm= >3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking/= >tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3te= >sts-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract= >

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-mi= >n-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-r= >eport-SOLgnu-*.html

TESTHOSTNAME=3Dniagara
WS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-57-02
LASTREL=3D5.4.0
INSTROOT_REL=3D= >/homes/hosking/work/cm3-inst/niagara/rel-5.4.0
INSTROOT_POK=3D/homes/hos= >king/work/cm3-inst/niagara/prev-ok
INSTROOT_LOK=3D/homes/hosking/work/cm= >3-inst/niagara/last-ok
INSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niaga= >ra/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET=3DSOLgnu
BINDISTMIN=3D/ho= >mes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
CM3CVSSERVER=3Dbirch.ele= >gosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvs
BINDISTMIN_NAME= >=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN=3D/homes/hosking/work/cm3-m= >in-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.elegosoft.com..
ssh birch= >.elegosoft.com ok
Building cm3.
Tinderbox Tree:  =3B =3B"cm3"= >
Buildname:  =3B =3B =3B =3B =3B =3B =3B"SOL= >gnu SunOS 5.10 niagara lastok-build"

creating log file /tmp/build-cm= >3-20081217-065704-RGaGCn/log.txt

---

checkout=2C compile and = >test of cm3 ...
2008.12.17 06:57:04 -- checkout in progress.
[start c= >heckout 2008.12.17 06:57:06]
cd /tmp/build-cm3-20081217-065704-RGaGCn/bu= >ild
cvs return value: 0
[end checkout 2008.12.17 07:16:18]
CHECKOU= >T_RETURN =3D 0
--
2008.12.17 07:16:21 -- compile in progress.
[sta= >rt compile 2008.12.17 07:16:21]
compile return value: 0
[end compile = >2008.12.17 08:01:31]
COMPILE_RETURN =3D 0
2008.12.17 08:01:38 -- test= >s in progress.
[start run-tests 2008.12.17 08:01:38]
cd /tmp/build-cm= >3-20081217-065704-RGaGCn/build
[end run-tests 2008.12.17 08:01:38]
TE= >STS_RETURN =3D 0
2008.12.17 08:01:38 -- checkout=2C compile and test run= > done.

---

removing build tree /tmp/build-cm3-20081217-065704= >-RGaGCn ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/nia= >gara-*

cleaning regression test log files...
/homes/hosking/tmp/c= >m3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking= >/tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3t= >ests-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extrac= >t

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-m= >in-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-= >report-SOLgnu-*.html

done.

PAN>

V>

body> >= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_-- From jay.krell at cornell.edu Thu Dec 18 13:25:48 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 12:25:48 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: > Do Cygwin users really care about all that stuff? Could be that some do, some don't. > The point of using Modula-3 on Cygwin But is "Cygwin" the point? Would Interix or UWin suffice? Maybe. Or just accepting forward slashes and providing a 64bit longint? Maybe. And what does "Cygwin" mean?It is many things. Maybe the point is to use X Windows? Not likely imho, but maybe. I hadn't considered this at first myself, but someone (Olaf?) asked for it. Or pthreads? Again not likely, but maybe. Or LONGINT? Maybe. Or Posix paths? ie. form and run command lines from bash. This seems to be the most popular reason. The "run" part, not a problem, the "form" part somewhat related to the path issue. Or to use a "super" optimizing compiler instead of just a fast "moderately" optimizing compiler? Or to use something you have more source to? i.e. gcc/ld/newlib. Or to interop with gcc-specific code? You can use a mix of compilers and interop, Cygwin, Visual C++, and the integrated backend all use the same object file format, but maybe it is easier to use one toolset more, than to mix. Or to use forward slashes in paths? You can do that with regular NT386 now, as you always could with the kernel32.dll functions. "Posix paths" vs. "forward slashes" not entirely the same thing, but almost. Unfortunately Win32 is not consistent here, like the file open dialogs and I think shlwapi.dll don't accept forward slashes. Arguably bugs. Or to interop with open/read/close/stat? You could always already do that too. Those functions do all exist in msvcr*.dll. Or to interop with very Berkeley socket-specific code? Maybe. I think Windows sockets are very very similar to Berkely/Posix sockets, but not quite the same. For example, I think an "fd set" is an array of opaque pointers, not a bit set. Or to link (again, interop) with code calling fork? Or because you want a better cross-build story? That is, you can build/run the Cygwin runtime, compiler, and linker on any other host, whereas the NT386 linker is only runnable on NT386. Or because you are familiar with gdb? Or due to issues around freedom of beer or liberty? Visual C++ has varied in its free-beer-ness. What is the point of the MinGWin project? I think it is about using autotools and gcc, but not sure. And producing binaries with "more flexible licensing" (no cygwin1.dll) One reason I got this stuff working is becauseI saw it as a shortcut to 64bit LONGINT support. I find Cygwin largely "cumbersome".Just usings its command line is slow.Slow to edit. gcc is slow.But the cross build story is very good.You know, I use Cygwin so that Windows can target Modula-3 to other systems. You know, to build m3cg. MinGWin I think suffices, but I got the impression at some point that Cygwin was better maintained than MinGWin. There are many factors.Modula-3 was already usable by Cygwin users, just as Visual C++ already is. The Modula-3 libraries shield you from almost all of this, if you stick to them.If you don't interop with much C code, if don't need a 64bit LONGINT, if thefast integrated backend generates efficient enough code for you, etc.The "external" interface really seems largely the form of paths. ...I /think/ most of the point to Cygwin to a lot of people, at some point, was to have a free beer C/C++ compiler. But again, this is only some people, and there are now other options. Digital Mars and Open Watcom are also pretty good. Digital Mars doesn't come with source. Open Watcom does. eBay provides pretty cheap stuff too. C/C++ compilers have lost their value as other programming languages (Java, JavaScript, C#, Perl, Python) have risen and sometimes been free (all free-beer, somewhat free-freedom/GPL). Somewhat due to the very "safety" issues that Modula-3 solves. Modula-3 I think was way ahead of its time. The world has mostly caught up, though Modula-3 is still a fairly unique hybrid, in terms of compiling directly to native code, and offering optional safety within one language. (C# also has optional safety within one language). If you only want safety and can sacrifice perf and/or generality and/or native code.. But again, there are many factors, many characteristics that define "Cygwin". - Jay> To: jay.krell at cornell.edu> Date: Wed, 17 Dec 2008 19:10:15 -0800> From: mika at async.caltech.edu> CC: jkrell at elego.de; m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > Do Cygwin users really care about all that stuff?> > The point of using Modula-3 on Cygwin is to turn a yucky Windows> machine as close as possible into a friendly Unix environment...> not to use a bunch of Microsoft tools and libraries---I would have> thought that if you wanted to use Visual X-Y-Z then you wouldn't> be using Cygwin but would be using the native Windows environment?> > So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> occasionally use on Windows uses Windows threads and has its own> compiler back-end. But it's kind of cool too and since it's not> really distinguishable from user-level threads and gcc I just think> "great" and get on with it. In fact I'm not even sure it uses> the integrated back end now that I think about it.> > I think what I'm saying is that almost no Cygwin users are terribly> interested in Microsoft interfaces, and I would bet most Cygwin> users aren't even aware of the existence of most Microsoft tools.> They (we) want as little to do with Windows as possible....> > Mika> > Jay writes:> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/plain; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >From: TonyTo: jay> >I would have thought of CygWin as a POSIX platform that just happens to hav=> >e Windows underneath. Did we go round this roundabout before?> >> >> >> >> >> >It's more complicated than that.> >It isn't just one bit.> >There's a few factors=2C that people care more and less about.> > GUI library -- Win32 or X> > thread library -- pthreads or Win32 (or user threads)=20> > file system path representation -- forward slashes or backward slashes => > This seems to be what people care most about -- to see forward slashes.> > C compiler -- Visual C++ cl or Cygwin/GNU gcc> > linker -- Visual C++ link or Cygwin/GNU ld=20> > C runtime -- msvcr*.dll or cygwin1.dll=20> > integrated backend or not The integrated backend is way faster=2C but=> > doesn't yet support 64bit LONGINT.> > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/=> >pdb> >Many of these factors are independent of each other=2C but not entirely.> >Compiler/linker directly lead to which debugger you can use=2C since they => >each use different formats.> >Well=2C you can use either debugger=2C but..without symbols..at least for s=> >ome modules.> >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa=> >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. => >This is probably fixable at least in one combination withan alias (ability => >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN=> >U ld understands aliases).> >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the=> >ir linker somehow accepts them.> >=20> >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by=> >te structs by value to __stdcall functions=2C the function name isn't mangl=> >ed correctly and linking fails.> >It is a bit of a combinatorial explosion.As the system stands=2C you can co=> >ntrol each bit in the config file=2C andmultiple combinations might work=2C=> > though I don't run any combinatorial tests.> >Three combinations are "identified" (named=2C spoken of) and have had some => >development testing.NT386=2C NT386MINGNU (which I think should be called I3=> >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN).> >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that=> >'s the name).Interix and UWIN being two other very viable Posix layers on W=> >indows.Interix has been renamed a few times. It is called "SFU" -- services=> > for Unix=2C and I don't know=2Clike SUA -- services for unix applications=> >=2C and I think there's another name.It is a free download from Microsoft f=> >or x86 Windows < Vista=2C and is an optional component in Vista.It has its => >pluses and minuses..not now.> >Anyway=2C in conversation here=2C one of the main things that people DO NOT=> > care aboutis which thread library is used.> >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h=> >ave used Win32 threads.> >And maybe never had any GUI?> >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres=> >h go at debugging it would pan out..but I'm really not too interested..> >I also have no interest in user threads. I believe they are never warranted=> >=2C excepton old systems that don't have kernel threads=2C like the hypothe=> >tical DJGPP port.> >I never brought up the issue of serial ports.I ASSUMED that almost nobody u=> >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem=> >ented using Cygwin or Win32=2C as long as they work.> >I ASSUME there is no interop issue=2C like getting an int fd from the seria=> >l portand passing it to select() or close=2C or getting a Win32 HANDLE and => >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly.> >I didn't look into if Cygwin is complete enough to implement them.I conside=> >r serial ports to be kind of out on the fringe..that it wouldn't be surpris=> >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui=> >lding minicom on some non-Linux system(s) recently and it failed miserably.=> > I don't know if there is even much Posix portability option for serial por=> >t programming.> >Sure..I debug over a serial port=2C and lately have machines I can't get vi=> >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us=> >e a serial consolefor initial install (recent achievement :) still haven't => >installed over network..)=2C but that's still imho a big step away from wri=> >ting code that uses the serial port.(And NetBSD doesn't support the local v=> >ideo on SGI O2=2C but I'm running OpenBSD there for now.)> >=20> >Anyway=2C there are a few ways to look at this issue.More and less trivial.> >A trivial view is: Get the filtering to work. Build serial on all platform=> >s but Cygwin. Or see if the Posix code will work.> > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p=> >rogrammers is not 0=2C but could be. I actually don't think anyone is using=> > Cygwin x Modula-3.> >Another view is: Get the Win32 serial code to work on Cygwin. The code is=> > already there=2C and buildable=2C but building in a system that also build=> >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio=> >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr=> >ovide the File interface=2C and FileWin32.m3 to provide just its own spec=> >ific FileWin32 interface. I think this is a simple transform of the code.> >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma=> >ybe I'll get the truncation I deserve. :)> > - Jay=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/html; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >> >> >> >
> >From: Tony
To: jay

> >
I would have thought of CygWin as a POSIX platform that just happens t=> >o have Windows underneath.  =3BDid we go round this roundabout before?<=> >/DIV>> >
>12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND=> >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS=> >E: separate">> >
>e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C=> >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S=> >PACING: normal=3B BORDER-COLLAPSE: separate"> >pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n=> >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B => >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN=> >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no=> >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T=> >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S=> >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet=> >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B=> > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate=> >"> >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT=> >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: => >separate"> >ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX=> >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO=> >LLAPSE: separate"> >0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0=> >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B=> >ORDER-COLLAPSE: separate">> >

 >V>
> >
> >
> FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T=> >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-=> >COLLAPSE: separate">> >
>>
It's more complicated than that.
> >
>>
It isn't just one bit.
> >
>>
There's a few factors=2C that people care more and less about.
> >
>>
 =3B GUI library =3B -- Win32 or X
> >
>>
 =3B thread library =3B -- pthreads or Win32 (or user threads)=> >
> >
>>
 =3B file system path representation -- forward slashes or backwar=> >d slashes
 =3B =3B =3B This seems to be what people care mos=> >t about -- to see forward slashes.
> >
>>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
> >
>>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
> >
>>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
> >
>>
 =3B integrated backend or not
 =3B =3B =3B The in=> >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT. >IV>> >
>>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi=> >suaStudio/CodeView/pdb
> >
>>
Many of these factors are independent of each other=2C but not entirel=> >y.
> >
>>
Compiler/linker directly lead to which debugger you can use=2C since >R> =3B they each use different formats.
> >
>>
Well=2C you can use either debugger=2C but..without symbols..at least => >for some modules.
> >
>>
The Cygwin C runtime and Visual C++ linker disagree
on the name of => >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual=> > C++ linker. This is probably fixable at least in one combination with
a=> >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't=> >
know if Cygwin/GNU ld understands aliases).
> >
>>
I have also found that Cygwin produces incorrect
import .libs=2C pe=> >rhaps their linker somehow accepts them.
> >
>> =3B
> >
>>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b=> >yte structs by value to __stdcall functions=2C the function name isn't mang=> >led correctly and linking fails.
> >
>>
It is a bit of a combinatorial explosion.
As the system stands=2C y=> >ou can control each bit in the config file=2C and
multiple combinations => >might work=2C though I don't run any combinatorial tests.
> >
>>
Three combinations are "identified" (named=2C spoken of) =3Band ha=> >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho=> >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called=> > I386_CYGWIN).
> >
>>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if=> > that's the name).
Interix and UWIN being two other very viable Posix la=> >yers on Windows.
Interix has been renamed a few times. It is called "SFU=> >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u=> >nix applications=2C and I think there's another name.
It is a free downl=> >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo=> >nent in Vista.
It has its pluses and minuses..not now.
> >
>>
Anyway=2C in conversation here=2C one of the main things that people D=> >O NOT care about
is which thread library is used.
> >
>>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi=> >ght have used Win32 threads.
> >
>>And maybe never had any GUI?
> >
>>
I tried getting pthreads to work=2C but gave up debugging it.
Perha=> >ps a fresh go at debugging it would pan out..but I'm really not too interes=> >ted..
> >
>>
I also have no interest in user threads. I believe they are never warr=> >anted=2C except
on old systems that don't have kernel threads=2C like th=> >e hypothetical DJGPP port.
> >
>>
I never brought up the issue of serial ports.
I ASSUMED that almost=> > nobody uses them=2C and anyone that does use them=2C doesn't
care if th=> >ey are implemented using Cygwin or Win32=2C as long as they work.
> >
>>I ASSUME there is no interop issue=2C like getting an int fd from the seri=> >al port
and passing it to select() or close=2C or getting a Win32 HANDLE=> > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c=> >ertainly.
> >
>>
I didn't look into if Cygwin is complete enough to implement them.
=> >I consider serial ports to be kind of out on the fringe..that it wouldn't b=> >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I => >tried building minicom on some non-Linux system(s) recently and it failed m=> >iserably. I don't know if there is even much Posix portability option for s=> >erial port programming.
> >
>>
Sure..I debug over a serial port=2C and lately have machines I can't g=> >et video
out of (Itanium) or even seem to have no video hardware (Sun)=> >=2C so I use a serial console
for initial install (recent achievement :)=> > still haven't installed over network..)=2C but that's still imho a big ste=> >p away from writing code that uses the serial port.
(And NetBSD doesn't => >support the local video on SGI O2=2C but I'm running OpenBSD there for now.=> >)
> >
>> =3B
> >
>>
Anyway=2C there are a few ways to look at this issue.
More and less=> > trivial.
> >
>>
A trivial view is:
 =3B Get the filtering to work. Build serial=> > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code => >will work.
> >
>> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria=> >l port programmers is not 0=2C but could be. I actually don't think anyone => >is using Cygwin x Modula-3.
> >
>>
Another view is:
 =3B Get the Win32 serial code to work on Cygw=> >in.
 =3B The code is already there=2C and buildable=2C but building => >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula=> >-3 rules=2C you end up with two implementations of the File interface.
&=> >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte=> >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci=> >fic FileWin32 interface.
 =3B I think this is a simple transform of => >the code.
> >
>>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor=> >ry.
Maybe I'll get the truncation I deserve. :)
> >
>>
 =3B- Jay
> >=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Thu Dec 18 21:46:26 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 18 Dec 2008 15:46:26 -0500 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: <494A6FFD.1E75.00D7.1@scires.com> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel about it and Modula-3. Mika has a point that if you want to create an environment on Windows that is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are looking for. On the other hand, I am in a camp of folks who don't mind using the Microsoft tools and want Modula-3 to work out of the box on Windows without having to add any other non-Microsoft stuff (like Cygwin). I think it is great for CM3 to be able to support both these camps! Thus, you will find me always advocating for keeping the "CM3 on Windows variant" where it will work on Windows using just the Microsoft tools. I am happy with the use of Windows threads and the native backend and use of the MS Visual Studio linker. Regards, Randy Coleburn >>> Mika Nystrom 12/17/2008 10:10 PM >>> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 22:21:07 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 18 Dec 2008 13:21:07 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> Message-ID: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Yes, I don't think the world is a terrible place because people like you exist :-) I just think that Cygwin users generally have modest requirements. They're just trying to get something---anything!---to work on Windows, and if they want to do anything that's really Windows-specific I think they/we understand they/we may have to make the effort to learn a bit about Windows and use the native environment. What it adds up to, in my opinion, is that one shouldn't screw things up in the native Windows port (or in the Unix ports) just to satisfy Cygwin users, who really don't care if Windows feature XYZ is available. How many people actually use Modula-3 on Cygwin? I do... and I just use it so I can run programs I developed on Unix on random Windows machines when the owners of those machines have made the bizarre (to me) decision to run Windows... If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y than it is now. For instance, the version I use has Windows file-locking semantics, i.e., default readers-and-writers locking on all open files, which can be quite annoying when porting from Unix. It also uses a different epoch for Time.T. Not sure if the current version has these features... Mika "Randy Coleburn" writes: >--=__Part6F4761B2.0__= >Content-Type: text/plain; charset=US-ASCII >Content-Transfer-Encoding: quoted-printable > >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >about it and Modula-3. >=20 >Mika has a point that if you want to create an environment on Windows that = >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >looking for. >=20 >On the other hand, I am in a camp of folks who don't mind using the = >Microsoft tools and want Modula-3 to work out of the box on Windows = >without having to add any other non-Microsoft stuff (like Cygwin). >=20 >I think it is great for CM3 to be able to support both these camps! >=20 >Thus, you will find me always advocating for keeping the "CM3 on Windows = >variant" where it will work on Windows using just the Microsoft tools. I = >am happy with the use of Windows threads and the native backend and use of = >the MS Visual Studio linker. >=20 >Regards, >Randy Coleburn > >>>> Mika Nystrom 12/17/2008 10:10 PM >>> >Do Cygwin users really care about all that stuff? > >The point of using Modula-3 on Cygwin is to turn a yucky Windows >machine as close as possible into a friendly Unix environment... >not to use a bunch of Microsoft tools and libraries---I would have >thought that if you wanted to use Visual X-Y-Z then you wouldn't >be using Cygwin but would be using the native Windows environment? > >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still >occasionally use on Windows uses Windows threads and has its own >compiler back-end. But it's kind of cool too and since it's not >really distinguishable from user-level threads and gcc I just think >"great" and get on with it. In fact I'm not even sure it uses >the integrated back end now that I think about it. > >I think what I'm saying is that almost no Cygwin users are terribly >interested in Microsoft interfaces, and I would bet most Cygwin >users aren't even aware of the existence of most Microsoft tools. >They (we) want as little to do with Windows as possible.... > > Mika > >Jay writes: >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >>Content-Type: text/plain; charset=3D"iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >>From: TonyTo: jay >>I would have thought of CygWin as a POSIX platform that just happens to = >hav=3D >>e Windows underneath. Did we go round this roundabout before? >> >> >> >> >> >>It's more complicated than that. >>It isn't just one bit. >>There's a few factors=3D2C that people care more and less about. >> GUI library -- Win32 or X >> thread library -- pthreads or Win32 (or user threads)=3D20 >> file system path representation -- forward slashes or backward slashes = > =3D >> This seems to be what people care most about -- to see forward slashes. >> C compiler -- Visual C++ cl or Cygwin/GNU gcc >> linker -- Visual C++ link or Cygwin/GNU ld=3D20 >> C runtime -- msvcr*.dll or cygwin1.dll=3D20 >> integrated backend or not The integrated backend is way faster=3D2C = >but=3D >> doesn't yet support 64bit LONGINT. >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView= >/=3D >>pdb >>Many of these factors are independent of each other=3D2C but not = >entirely. >>Compiler/linker directly lead to which debugger you can use=3D2C since = >they =3D >>each use different formats. >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least = >for s=3D >>ome modules. >>The Cygwin C runtime and Visual C++ linker disagreeon the name of = >__ImageBa=3D >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ = >linker. =3D >>This is probably fixable at least in one combination withan alias = >(ability =3D >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if = >Cygwin/GN=3D >>U ld understands aliases). >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps = >the=3D >>ir linker somehow accepts them. >>=3D20 >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing = >8 by=3D >>te structs by value to __stdcall functions=3D2C the function name isn't = >mangl=3D >>ed correctly and linking fails. >>It is a bit of a combinatorial explosion.As the system stands=3D2C you = >can co=3D >>ntrol each bit in the config file=3D2C andmultiple combinations might = >work=3D2C=3D >> though I don't run any combinatorial tests. >>Three combinations are "identified" (named=3D2C spoken of) and have had = >some =3D >>development testing.NT386=3D2C NT386MINGNU (which I think should be = >called I3=3D >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= From jay.krell at cornell.edu Thu Dec 18 23:12:12 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 22:12:12 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y Well, here I can cop-out and say: "Hey, it just uses Cygwin". Other than a possible avenue for the serial code. And other than threads. > Unix. It also uses a different epoch for Time.T. Not sure if I'm pretty sure that's not the case now. It very likely uses 31bit seconds since 1970, and will break around 2038. The hypothetical AMD64_CYGWIN would likely use 63bit seconds since 1970, so better. msvcr*.dll offers 63bits seconds since 1970, you configure it with a #define. (I say 31 or 63, because they are signed numbers probably, but negative numbers aren't considered valid. They really should be unsigned, and with a much earlier epoch...I realize files don't exist before these times, but it's nice to have one time format apply to more than just files, and there's certainly a legitimate need to represent earlier times..) Oh..well, yeah, I did name files foo.lib and not libfoo.a. File naming was another question that came up, along the lines of "What is Cygwin?". People said they didn't care. Similarly, executables named "foo" or "foo.exe"? .dlls named "foo.so" or "foo.dll". Cygwin actually uses foo.dll (or libfoo.dll, er..cygfoo.dll actually) and foo.exe. foo.so works fine, and "foo" works fine, at least on NT. bash will run it. cmd will run it if you add "." to %PATHEXT%. Apparently Win9x doesn't like it, and Cygwin historically supported Win9x (still today, but not for much longer, not with the next major release). So here you can see, that some typical Posix characteristics are not followed by Cygwin. Interix actually does use extensionless executables. You can see the evidence in taskmgr for example. I had it using foo.so (libfoo.so?) and "foo" for a bit, but since people didn't care, and Cygwin didn't do that, I put it back. It was pretty easy to go back and forth, like by fiddling with the "naming conventions" setting. (Which, it turns out, I think, was never really implemented correctly -- there are host and target conventions and I think they were confused.) Wait a sec, this does remind me. The cygfoo.dll convention has the benefit of being able to mix stuff in the same directory or along the same %PATH%. I thought that not useful, since .exes would still clash in name, but maybe? I will put serial back for non-Cygwin, really. And I'll fix SOLsun/gnu. Soon. - Jay> To: rcoleburn at scires.com> To: m3devel at elegosoft.com> Date: Thu, 18 Dec 2008 13:21:07 -0800> From: mika at async.caltech.edu> Subject: Re: [M3devel] serial/Cygwin> > Yes, I don't think the world is a terrible place because people like> you exist :-)> > I just think that Cygwin users generally have modest requirements.> They're just trying to get something---anything!---to work on> Windows, and if they want to do anything that's really Windows-specific> I think they/we understand they/we may have to make the effort to> learn a bit about Windows and use the native environment.> > What it adds up to, in my opinion, is that one shouldn't screw> things up in the native Windows port (or in the Unix ports) just> to satisfy Cygwin users, who really don't care if Windows feature> XYZ is available.> > How many people actually use Modula-3 on Cygwin? I do... and I> just use it so I can run programs I developed on Unix on random> Windows machines when the owners of those machines have made the> bizarre (to me) decision to run Windows...> > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> than it is now. For instance, the version I use has Windows> file-locking semantics, i.e., default readers-and-writers locking> on all open files, which can be quite annoying when porting from> Unix. It also uses a different epoch for Time.T. Not sure if> the current version has these features...> > Mika> > "Randy Coleburn" writes:> >--=__Part6F4761B2.0__=> >Content-Type: text/plain; charset=US-ASCII> >Content-Transfer-Encoding: quoted-printable> >> >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >about it and Modula-3.> >=20> >Mika has a point that if you want to create an environment on Windows that => >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >looking for.> >=20> >On the other hand, I am in a camp of folks who don't mind using the => >Microsoft tools and want Modula-3 to work out of the box on Windows => >without having to add any other non-Microsoft stuff (like Cygwin).> >=20> >I think it is great for CM3 to be able to support both these camps!> >=20> >Thus, you will find me always advocating for keeping the "CM3 on Windows => >variant" where it will work on Windows using just the Microsoft tools. I => >am happy with the use of Windows threads and the native backend and use of => >the MS Visual Studio linker.> >=20> >Regards,> >Randy Coleburn> >> >>>> Mika Nystrom 12/17/2008 10:10 PM >>>> >Do Cygwin users really care about all that stuff?> >> >The point of using Modula-3 on Cygwin is to turn a yucky Windows> >machine as close as possible into a friendly Unix environment...> >not to use a bunch of Microsoft tools and libraries---I would have> >thought that if you wanted to use Visual X-Y-Z then you wouldn't> >be using Cygwin but would be using the native Windows environment?> >> >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> >occasionally use on Windows uses Windows threads and has its own> >compiler back-end. But it's kind of cool too and since it's not> >really distinguishable from user-level threads and gcc I just think> >"great" and get on with it. In fact I'm not even sure it uses> >the integrated back end now that I think about it.> >> >I think what I'm saying is that almost no Cygwin users are terribly> >interested in Microsoft interfaces, and I would bet most Cygwin> >users aren't even aware of the existence of most Microsoft tools.> >They (we) want as little to do with Windows as possible....> >> > Mika> >> >Jay writes:> >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >>Content-Type: text/plain; charset=3D"iso-8859-1"> >>Content-Transfer-Encoding: quoted-printable> >>> >>> >>> >>From: TonyTo: jay> >>I would have thought of CygWin as a POSIX platform that just happens to => >hav=3D> >>e Windows underneath. Did we go round this roundabout before?> >>> >>> >>> >>> >>> >>It's more complicated than that.> >>It isn't just one bit.> >>There's a few factors=3D2C that people care more and less about.> >> GUI library -- Win32 or X> >> thread library -- pthreads or Win32 (or user threads)=3D20> >> file system path representation -- forward slashes or backward slashes => > =3D> >> This seems to be what people care most about -- to see forward slashes.> >> C compiler -- Visual C++ cl or Cygwin/GNU gcc> >> linker -- Visual C++ link or Cygwin/GNU ld=3D20> >> C runtime -- msvcr*.dll or cygwin1.dll=3D20> >> integrated backend or not The integrated backend is way faster=3D2C => >but=3D> >> doesn't yet support 64bit LONGINT.> >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView=> >/=3D> >>pdb> >>Many of these factors are independent of each other=3D2C but not => >entirely.> >>Compiler/linker directly lead to which debugger you can use=3D2C since => >they =3D> >>each use different formats.> >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least => >for s=3D> >>ome modules.> >>The Cygwin C runtime and Visual C++ linker disagreeon the name of => >__ImageBa=3D> >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ => >linker. =3D> >>This is probably fixable at least in one combination withan alias => >(ability =3D> >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if => >Cygwin/GN=3D> >>U ld understands aliases).> >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps => >the=3D> >>ir linker somehow accepts them.> >>=3D20> >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing => >8 by=3D> >>te structs by value to __stdcall functions=3D2C the function name isn't => >mangl=3D> >>ed correctly and linking fails.> >>It is a bit of a combinatorial explosion.As the system stands=3D2C you => >can co=3D> >>ntrol each bit in the config file=3D2C andmultiple combinations might => >work=3D2C=3D> >> though I don't run any combinatorial tests.> >>Three combinations are "identified" (named=3D2C spoken of) and have had => >some =3D> >>development testing.NT386=3D2C NT386MINGNU (which I think should be => >called I3=3D> >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Dec 18 23:03:06 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 18 Dec 2008 16:03:06 -0600 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: <494AC89A.4040601@wichita.edu> I agree that it is highly desirable to have both native windows and Cygwin variants of Modula-3. But I also think that it is unrealistic to try to support a complete or even partial Cartesian product of all the many separate options Jay enumerates. Without debating whether it might be desirable, it's just way too much work when there are other, more pressing things to do with our limited support time. Mix and match is just entirely unrealistic. By the same reasoning, I have given up on the similar ideal of making m3gdb handle a linked-together mix of code from different Modula-3 compilers: PM3, CM3, standalone backend, different versions of gcc-derived back end, etc. I advocate a Cygwin target that is as Unix-like as possible, in as many respects as possible, and a native Windows target that is as Windows-like as possible in as many respects as possible. No more. Mika Nystrom wrote: > Yes, I don't think the world is a terrible place because people like > you exist :-) > > I just think that Cygwin users generally have modest requirements. > They're just trying to get something---anything!---to work on > Windows, and if they want to do anything that's really Windows-specific > I think they/we understand they/we may have to make the effort to > learn a bit about Windows and use the native environment. > > What it adds up to, in my opinion, is that one shouldn't screw > things up in the native Windows port (or in the Unix ports) just > to satisfy Cygwin users, who really don't care if Windows feature > XYZ is available. > > How many people actually use Modula-3 on Cygwin? I do... and I > just use it so I can run programs I developed on Unix on random > Windows machines when the owners of those machines have made the > bizarre (to me) decision to run Windows... > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y > than it is now. For instance, the version I use has Windows > file-locking semantics, i.e., default readers-and-writers locking > on all open files, which can be quite annoying when porting from > Unix. It also uses a different epoch for Time.T. Not sure if > the current version has these features... > > Mika > > "Randy Coleburn" writes: >> --=__Part6F4761B2.0__= >> Content-Type: text/plain; charset=US-ASCII >> Content-Transfer-Encoding: quoted-printable >> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >> about it and Modula-3. >> =20 >> Mika has a point that if you want to create an environment on Windows that = >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >> looking for. >> =20 >> On the other hand, I am in a camp of folks who don't mind using the = >> Microsoft tools and want Modula-3 to work out of the box on Windows = >> without having to add any other non-Microsoft stuff (like Cygwin). >> =20 >> I think it is great for CM3 to be able to support both these camps! >> =20 >> Thus, you will find me always advocating for keeping the "CM3 on Windows = >> variant" where it will work on Windows using just the Microsoft tools. I = >> am happy with the use of Windows threads and the native backend and use of = >> the MS Visual Studio linker. >> =20 >> Regards, >> Randy Coleburn >> From jay.krell at cornell.edu Fri Dec 19 12:11:54 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 11:11:54 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <494AC89A.4040601@wichita.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> <494AC89A.4040601@wichita.edu> Message-ID: Pretty much agreed. The implementation is factored along the lines I describe, but I don't test the cross product. I think there might be one other interesting middle ground -- a target that uses the gcc backend but is otherwise Windows-like..oh, and probably GNU/gcc and GNU/ld, else you can't debug. This is analogous to the "MinGWin" project, which is real and works, it offers gcc, that can I believe build itself, but which generally produces binaries with no extra dependency or "strange" :) behavior. There is a config file for this called NT386MINGNU, though I think it should be called I386_MINGWIN. The system was able to build itself, and getting it to work was a subset of Cygwin -- they shared an important bug. This port can't use Trestle currently due to a minor bug in cm3cg. The benefits of this middle ground are that you get 64bit LONGINT already, and you might get highly optimized code instead of only moderately optimized code. Personally, I almost never ask gcc to optimize. You'll also probably get "AMD64_MINGW" before "AMD64_NT". Neither all that soon, but I know MinGWin/AMD64 is very far along or even done. (Other avenues to avoid updating the backend are C_NT and LLVM_NT, but these are so far just fantasies.) > The implementation is factored along the lines I describe And that is an exaggeration. It is mostly the config file that is factored such. Within the Modula-3 compiler, the only distinction is the "backend mode". The jmpbuf size is chosen I think to be the maximum of the two C runtimes. In the distant distant future, the compiler could also distinguish exception handling codegen strategies. There are several to chose from, like 3 or 4 for x86, 2 for the rest. - setjmp/longjmp -- as it stands now, or interoperable with gcc's similar - "elf/dwarf/whatever" -- range tables, stack walking, gcc supports this these days - NT/x86 style -- builds a linked list through a special per-thread/fiber location conceptually like setjmp/longjmp, but highly optimized. - NT/other -- basically like "elf", but perhaps the relevant data and runtime is different in the details - Jay> Date: Thu, 18 Dec 2008 16:03:06 -0600> From: rodney.bates at wichita.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > I agree that it is highly desirable to have both native windows and Cygwin> variants of Modula-3. But I also think that it is unrealistic to try to> support a complete or even partial Cartesian product of all the many> separate options Jay enumerates. Without debating whether it might be> desirable, it's just way too much work when there are other, more pressing> things to do with our limited support time.> > Mix and match is just entirely unrealistic. By the same reasoning, I> have given up on the similar ideal of making m3gdb handle a linked-together> mix of code from different Modula-3 compilers: PM3, CM3, standalone> backend, different versions of gcc-derived back end, etc.> > I advocate a Cygwin target that is as Unix-like as possible, in as many> respects as possible, and a native Windows target that is as Windows-like> as possible in as many respects as possible. No more.> > > > Mika Nystrom wrote:> > Yes, I don't think the world is a terrible place because people like> > you exist :-)> > > > I just think that Cygwin users generally have modest requirements.> > They're just trying to get something---anything!---to work on> > Windows, and if they want to do anything that's really Windows-specific> > I think they/we understand they/we may have to make the effort to> > learn a bit about Windows and use the native environment.> > > > What it adds up to, in my opinion, is that one shouldn't screw> > things up in the native Windows port (or in the Unix ports) just> > to satisfy Cygwin users, who really don't care if Windows feature> > XYZ is available.> > > > How many people actually use Modula-3 on Cygwin? I do... and I> > just use it so I can run programs I developed on Unix on random> > Windows machines when the owners of those machines have made the> > bizarre (to me) decision to run Windows...> > > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> > than it is now. For instance, the version I use has Windows> > file-locking semantics, i.e., default readers-and-writers locking> > on all open files, which can be quite annoying when porting from> > Unix. It also uses a different epoch for Time.T. Not sure if> > the current version has these features...> > > > Mika> > > > "Randy Coleburn" writes:> >> --=__Part6F4761B2.0__=> >> Content-Type: text/plain; charset=US-ASCII> >> Content-Transfer-Encoding: quoted-printable> >>> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >> about it and Modula-3.> >> =20> >> Mika has a point that if you want to create an environment on Windows that => >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >> looking for.> >> =20> >> On the other hand, I am in a camp of folks who don't mind using the => >> Microsoft tools and want Modula-3 to work out of the box on Windows => >> without having to add any other non-Microsoft stuff (like Cygwin).> >> =20> >> I think it is great for CM3 to be able to support both these camps!> >> =20> >> Thus, you will find me always advocating for keeping the "CM3 on Windows => >> variant" where it will work on Windows using just the Microsoft tools. I => >> am happy with the use of Windows threads and the native backend and use of => >> the MS Visual Studio linker.> >> =20> >> Regards,> >> Randy Coleburn> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Dec 19 19:25:52 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 18:25:52 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay From dabenavidesd at yahoo.es Sat Dec 20 01:09:41 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 20 Dec 2008 00:09:41 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Dear all: I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have. However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not? claim any more they are safe so that "fix" is not feasible. Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway?? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file. Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ),? that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation. There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that? Thanks in advance --- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/Cygwin Para: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25 > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Dec 20 08:19:02 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 19 Dec 2008 23:19:02 -0800 Subject: [M3devel] This disgusting TEXT business Message-ID: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Hello Modula-3ers, I have gone on the record before as not being very impressed by the Critical Mass implementation of TEXT. If you bother to read this email to the end, I will show you a simple program that runs 100 times slower when compiled with CM3 than the older PM3. TEXT is a very convenient built-in type, especially I think because it allows literals to be written with a lot less work than literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '\n' } Hence a lot of programs use TEXT liberally. Modula-3 also has a very syntactically convenient TEXT concatenation operator, &. I think it was Guy Steele who observed in one of the Lambda Papers that programmers tend to think syntactically concise notation is associated with efficient implementation. CM3 fails here, and sometimes horribly! I remember about five years ago we were writing a large number of VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" to CM3---I think one of us wanted to run the tools on a Macintosh. Most VLSI CAD tools really do nothing but process strings in various ways. Concatenation, splitting, those sorts of things. Think a large program written in a language without recursion but lots of hierarchical names. Compile unit x containing subunit y containing subunit z containing node n, giving name x.y.z.n . Well to make a long story short, the programmer with the Mac said that CM3 produced horribly slow-running binaries. Something with TEXTs, but we didn't investigate further. We gave up on migrating our group to CM3 and instead use PM3 to this day. The programmer with the Mac has since switched all his work to coding in C++ "in the style of Modula-3." And after seeing what I have seen now I cannot say I will voluntarily use CM3 for anything until we've got a solution. I would personally be happy to rip out everything that says Critical Mass TEXT on it and put back the SRC code. Here's the program I promised: MODULE Main; IMPORT Text, Fmt, IO, Time; CONST Count = 1000*1000; VAR checks := 0; start := Time.Now(); a : TEXT := ""; BEGIN FOR i := 0 TO 100 DO a := a & ":" & Fmt.Int(i) END; LOOP EVAL Text.Equal(a,a); INC(checks); IF checks > Count THEN IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / (Time.Now() - start)) & "\n"); start := Time.Now(); checks := 0 END END END Main. On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: virtual resident Text.Equals machine cycles size size per second compiler per char (approx) --------------------------------------------------------------- 11476K 2396K 13,274 CM3 770 1888K 960K 1,634,000 PM3 6.24 The culprit is TextCat.MyGetWideChars, which calls itself recursively. Here's a representative traceback: Starting program: /big/home/mika/test_text/FreeBSD4/test_text ^C Program received signal SIGINT, Interrupt. 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 111 TextCat.m3: No such file or directory. in TextCat.m3 (m3gdb) where #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) at Text.m3:66 #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 #10 0x08048b0d in Main (mode=1) at Main.m3:20 #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) at RTLinker.m3:122 #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at _m3main.mc:4 (m3gdb) Now you may object that perhaps people don't normally make TEXTs by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes they do. There's nothing to warn the user that that is a terribly bad thing to do. Well we know... we think.. it will lead to a lot of memory allocation overhead when you *make* the TEXT. But that it will infect the generated TEXT forevermore with this long chain of pointers is, to say the least, surprising. The reason I noticed this now is that I wrote a program that "watches" a directory of files (using FS.Iterate). It even uses a hash table to store the TEXTs. *Still* it is an incredible CPU hog (on Linux wth CM3). I think I just did path := dir & "/" & filename and that's enough to make it run 10-30 times slower than a PM3-compiled binary from the same source code (for suitably pathological choices of dir and filename). Does anyone have a better idea than just throwing away the CM3 TEXT implementation? Mika From hosking at cs.purdue.edu Sat Dec 20 09:26:40 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 20 Dec 2008 19:26:40 +1100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Hmm, are we just victims of poor implementation? Does anyone have the time to improve things? It would be possible to rip out CM3 TEXT and replace with PM3, but we'd lose WIDECHAR and WideText.T with that too. Not sure who that impacts. On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. > > If you bother to read this email to the end, I will show you a > simple program that runs 100 times slower when compiled with > CM3 than the older PM3. > > TEXT is a very convenient built-in type, especially I think because > it allows literals to be written with a lot less work than > > literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', > 'r', 'l', 'd', '!', '\n' } > > Hence a lot of programs use TEXT liberally. > > Modula-3 also has a very syntactically convenient TEXT concatenation > operator, &. > > I think it was Guy Steele who observed in one of the Lambda Papers > that > programmers tend to think syntactically concise notation is associated > with efficient implementation. > > CM3 fails here, and sometimes horribly! > > I remember about five years ago we were writing a large number of > VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" > to CM3---I think one of us wanted to run the tools on a Macintosh. > Most VLSI CAD tools really do nothing but process strings in various > ways. Concatenation, splitting, those sorts of things. Think a > large program written in a language without recursion but lots of > hierarchical names. Compile unit x containing subunit y containing > subunit z containing node n, giving name x.y.z.n . > > Well to make a long story short, the programmer with the Mac said > that CM3 produced horribly slow-running binaries. Something with > TEXTs, but we didn't investigate further. We gave up on migrating > our group to CM3 and instead use PM3 to this day. The programmer > with the Mac has since switched all his work to coding in C++ "in > the style of Modula-3." > > And after seeing what I have seen now I cannot say I will voluntarily > use CM3 for anything until we've got a solution. I would personally > be happy to rip out everything that says Critical Mass TEXT on it and > put back the SRC code. > > Here's the program I promised: > > MODULE Main; > IMPORT Text, Fmt, IO, Time; > > CONST > Count = 1000*1000; > > VAR > checks := 0; > start := Time.Now(); > a : TEXT := ""; > BEGIN > > FOR i := 0 TO 100 DO > a := a & ":" & Fmt.Int(i) > END; > > LOOP > EVAL Text.Equal(a,a); > INC(checks); > IF checks > Count THEN > IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / > (Time.Now() - start)) & "\n"); > start := Time.Now(); > checks := 0 > END > END > > END Main. > > On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo > E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: > > virtual resident Text.Equals machine cycles > size size per second compiler per char (approx) > --------------------------------------------------------------- > 11476K 2396K 13,274 CM3 770 > 1888K 960K 1,634,000 PM3 6.24 > > The culprit is TextCat.MyGetWideChars, which calls itself recursively. > Here's a representative traceback: > > Starting program: /big/home/mika/test_text/FreeBSD4/test_text > ^C > Program received signal SIGINT, Interrupt. > 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > 111 TextCat.m3: No such file or directory. > in TextCat.m3 > (m3gdb) where > #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) > at Text.m3:66 > #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 > #10 0x08048b0d in Main (mode=1) at Main.m3:20 > #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 > #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 > #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) > at RTLinker.m3:122 > #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at > _m3main.mc:4 > (m3gdb) > > Now you may object that perhaps people don't normally make TEXTs > by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes > they do. There's nothing to warn the user that that is a terribly > bad thing to do. Well we know... we think.. it will lead to a lot > of memory allocation overhead when you *make* the TEXT. But that > it will infect the generated TEXT forevermore with this long chain > of pointers is, to say the least, surprising. > > The reason I noticed this now is that I wrote a program that "watches" > a directory of files (using FS.Iterate). It even uses a hash table > to store the TEXTs. *Still* it is an incredible CPU hog (on Linux > wth CM3). I think I just did > > path := dir & "/" & filename > > and that's enough to make it run 10-30 times slower than a PM3- > compiled > binary from the same source code (for suitably pathological choices > of dir and filename). > > Does anyone have a better idea than just throwing away the CM3 TEXT > implementation? > > Mika From dragisha at m3w.org Sat Dec 20 09:00:54 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 09:00:54 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <1229760054.18256.134.camel@faramir.m3w.org> On Fri, 2008-12-19 at 23:19 -0800, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. If I ever saw an understatement here - then this is one :-) To make long story even shorter - what CM3 tried to do with their WIDECHAR bussiness and extra-fast concatenations was obsoleted very fast after (and some of it before) they put all that hell of code in their codebase. Whatever they thought to do - UTF-8 is doing with simple byte arrays, and string operations like array move are fastest non-trivial operations on CPUs since Z80. I hope we will see some end of it in times future. TIA. -- Dragi?a Duri? From dragisha at m3w.org Sat Dec 20 10:10:45 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 10:10:45 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Message-ID: <1229764245.18256.148.camel@faramir.m3w.org> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either slow, or produces results that make subsequent operations slow - is where problem is with this implementation. IMO, best solution would be to replace internal representation with UTF-8. For whom may be concerned with it - make some external widechar conversion routines available. That way - concat would be as fast as it can be made and other operations would be realistic - it is how almost everybody does their Unicode strings, after all. Almost everybody - excluding mobile industry and Microsoft :-), AFAIK. Compiler changes would make transparent conversion of literals. Everything else is already transparent with regard to internal representation. I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant to accept them as he did not believe C base routines were widespread enough. GNU world has no such reluctance. Everything is UTF8. If nobody can fill this job in next few weeks, I will probably have some time this winter. No promise, of course, but list will know :-). dd On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: > Hmm, are we just victims of poor implementation? Does anyone have the > time to improve things? It would be possible to rip out CM3 TEXT and > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > too. Not sure who that impacts. > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > > > Hello Modula-3ers, > > > > I have gone on the record before as not being very impressed by the > > Critical Mass implementation of TEXT. > > -- Dragi?a Duri? From jay.krell at cornell.edu Sat Dec 20 17:19:49 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 20 Dec 2008 16:19:49 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: My opinions.. UTF8 is a hack. It lets people do approximately nothing and claim they have done a lot. UTF8 is popular, but I'm still very skeptical of it. The best representation for Unicode is fixed size 16 bit characters, as Windows uses "everywhere". All the functions taking 8 bit characters are just thin wrappers over functions dealing with 16 bit characters. I believe NTFS stores 16 bit strings natively on disk. And so does FAT -- anything within a limited character set gets a regular old 8 bit 8.3 entry on disk. Anything "long", or with "other characters", or other forms like anything with two dots, I believe is stored as full 16 bit characters. In fact, I believe if you write an installable file system on Win9x, you deal with all 16 bit characters. What Windows does with 8 bit strings if of course much worse than UTF8. 8 bit characters on Windows are often considered "encoded in the current code page", with usually no record of the code page. I think probably the best thing to do with 8 bit characters is actually require them to be 7 bits. Or even 5 or 6 six bits, and conversion between them is just zero extension or 8 bit truncation. UTF8 is progress because it is, like, one universal code page. UTF8 is slightly muddied because Java defines it differently. Specifically I think there is an issue as to how to represent an embedded nul. There is also UTF7. I guess that's only useful in transports that aren't 8 bit clean. Is UTF a "transport" mechanism only, for an encoding to leave data in as much as possible? That is gray of course. "Entering" and "exiting" from "transport" is expensive when done in bulk, and avoided for perf. The best representation for a "string" is a length and an array of 16 bit characters, probably via a separate pointer, not rigth after the length. Arguably you need a 32 bit character since Unicde is, I believe, actually a 20 bit encoding. However here I'm willing to see things the other way. (ie: maybe hypocritical) 8 bit characters often suffice and it is probably worth being somewhat "dynamic". 8 bit bytes are also common, but should not be confused with characters and strings. Whatever functions any UTF8 code depends on, can be trivially implemented one's self, in portable C or portable Modula-3. How about.. I realized the Modula-3 code is "dynamic" and can deal with 8 or 16 bit characters. I didn't realize it concated strings via keeping around the pointers. This is actually a viable faster approach sometimes. They are called "ropes" in some contexts -- such as the SGI STL. I have to read the code, but..two obvious suggestions I am sympathetic to: 1) Leave things mostly alone, but add more functions people can call. Like String.Flatten, String.Cleanup, String.Seal. That would walk all the pointers and copy the data into one flat string. "Seal" is an exaggeration, since you could subsequently change the string. Problem with this approach of course is that any code that is slowed down, remains slowed down. You still have to do a little work to get back the perf. However, any code sped up by the current code remains sped up. 2) Always "flatten" upon concat. Remain dynamic in the representation, based on what characters are seen. Once a character above 127 is seen, the string is made wide. Any operation on a wide string that both returns a subset of it, AND has to anyway visit every character, can look for the opportunity to shorten it, IF a separate copy is made. Operations such as taking a prefix or suffix -- which can be made by just bumping the pointer or length, do NOT need to visit the characters and would not opportunistically narrow. Can taking a suffix like that work, with the garbage collector? Furthermore, the expression: foo & "a" & bar & "b" should be transformed, by the compiler, into one function call. If not currently, then "to do". This alone, with nothing else, might help a lot? Among the base-most string operations is not or shall not be "concat two strings", but rather "concat n strings". Hm. I think there's a third good option. Basically again looking to Windows. Windows has no one true string type..while internally Unicode is universal, all string functions are doubled up. For any given function foo, say for example CreateFile, there are actually two functions, CreateFileA and CreateFileW. To make a string literal in C or C++, you either say "foo" for an 8 bit string, or L"foo" for 16 bits. String length -- wcslen and strlen -- these names come from standard C and there are a bunch of them. pro: no existing code changes meaning con: no existing code changes meaning Anyone who needs unicode, needs to change their code, somehow. There is a static indirection mechanism. #ifdef UNICODE #define TCHAR WCHAR #define TEXT(a) L pasted to a #define CreateFile CreateFileW etc. #else #define TCHAR char #define a #define CreateFile CreateFileA etc. #endif You can write code to be portable either way. This was a dubious idea, because most code only compiles one way or the other. But it does let you migrate slowly -- keep 8 bit strings but write your code with the future in mind, and then later throw the switch. Personally I just hardcode everything WCHAR, CreateFileW, no illusions. Anyway, I bring this up to try to come up with a proposal along the lines of: - Take everything back to the way it was. - Offer some new way to support Unicode. but I'm not sure what the second part is. Is TEXT abstract enough that it can do double duty? What sort of interop is needed between code using narrow text and code using wide text? Must it be "automatic"? While I say UTF8 is a hack, I like something almost the same as Dragi?a says. Keep everything as 7 or 8 bit characters. Provide conversion routines. Make no assumptions internally that the characters are UTF8. Any internal string as array of char walking would just use individual bytes. Or a "decoding callback". ? Reject any text literal that isn't 7 or 8 bit clean. ? I don't know, I guess I'm very on the fence. Unicode is important and all, interop seems maybe important, it is hard to get anyone to change existing code..what CM3 did at least was seamless..in terms of no code change, well, er..perf change, and they barred direct access to the characters, which might still be good. Maybe you can ask for a direct pointer, of a specific type, and if you ask for the wrong type, you get NULL? ?? Now, this email contains no code to back it up. :) - Jay> From: dragisha at m3w.org> To: m3devel at elegosoft.com> Date: Sat, 20 Dec 2008 10:10:45 +0100> Subject: Re: [M3devel] This disgusting TEXT business> > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either> slow, or produces results that make subsequent operations slow - is> where problem is with this implementation.> > IMO, best solution would be to replace internal representation with> UTF-8. For whom may be concerned with it - make some external widechar> conversion routines available.> > That way - concat would be as fast as it can be made and other> operations would be realistic - it is how almost everybody does their> Unicode strings, after all. Almost everybody - excluding mobile industry> and Microsoft :-), AFAIK.> > Compiler changes would make transparent conversion of literals.> Everything else is already transparent with regard to internal> representation.> > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant> to accept them as he did not believe C base routines were widespread> enough. GNU world has no such reluctance. Everything is UTF8.> > If nobody can fill this job in next few weeks, I will probably have some> time this winter. No promise, of course, but list will know :-).> > dd> > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote:> > Hmm, are we just victims of poor implementation? Does anyone have the > > time to improve things? It would be possible to rip out CM3 TEXT and > > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > > too. Not sure who that impacts.> > > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote:> > > > > Hello Modula-3ers,> > >> > > I have gone on the record before as not being very impressed by the> > > Critical Mass implementation of TEXT.> > >> > -- > Dragi?a Duri? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Dec 20 19:20:12 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 19:20:12 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <1229797212.18256.157.camel@faramir.m3w.org> On Sat, 2008-12-20 at 16:19 +0000, Jay wrote: > My opinions.. > > UTF8 is a hack. > It lets people do approximately nothing and claim they have done a > lot. > UTF8 is popular, but I'm still very skeptical of it. I am very skeptical to skepticism without arguments. Extremelly skeptical when argument is "Windows does it...". > The best representation for Unicode is fixed size 16 bit characters, > as Windows > uses "everywhere". All the functions taking 8 bit characters are > just thin > wrappers over functions dealing with 16 bit characters. And both these facts/argument/whatever sounds scary, at least for me, me being a person of Modula-2/Modula-3 upbringing :-). UTF-8 has done everything it promised. UTF-8 is everywhere, except in Windows and mobile industry. This I/O conversion routines will do when cooperating with these worlds. As you pointed out - 16bit is not enoough for Unicode. UTF-8 covers all Unicode characters. If only this one is argument for Modula-3 Unicode representation in 2009 - it is enough for me. -- Dragi?a Duri? From roland.illig at gmx.de Sat Dec 20 19:38:01 2008 From: roland.illig at gmx.de (Roland Illig) Date: Sat, 20 Dec 2008 19:38:01 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <494D3B89.3020802@gmx.de> Dragi?a Duri? schrieb: > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. I object. UTF-8 is a good external representation format, but in memory, it is far more efficient to let all characters of a string have the same size. Java, for example, has defined a character to be "what fits into 16 bits". That's good enough in many situations. One of the situations it fails is when you want to represent Domino Tiles or Mahjong Tiles as characters. To give you some arguments against UTF-8 as in-memory representation: * How fast can you calculate the length of a string? (With length I mean the number of characters, not the memory size needed for storing them.) * Where does substring(5) start? Where does substring(10000) start, and how long does it take to find out? An argument for the UTF-8 representation is that it saves memory -- one byte for every ASCII character. Is this argument still as important as it has been 40 years ago? Roland From wagner at elegosoft.com Sat Dec 20 20:27:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 20 Dec 2008 20:27:55 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Hi everybody, I haven't followed all the discussions recently, but think it may help to comment on a few things here. IIRC the main reason for the current TEXT implementation was to become compatible with Java. Java strings and M3 strings should look the same for the embedded JVM written in M3. AFAIK this JVM has never been made open source nor is there any intention to do it. Still it may be a good idea to adhere to the TEXT representation if we ever intend to combine Java and M3 in one runtime. I don't think the implementation of the combined 8/16 bit TEXTs is very mature. I think that some improvements could be made. Has anybody looked into the details of the implementation? As it is already pointed out on the old CM3 notes on the web pages, some things are slower while others are now faster. I agree that the implementation is rather suboptimal for some common use cases. We'd need a set of use cases and tests agreed upon to really compare implementations though. Just one test for cat is not enough IMO. This should be the first step for an improvement. I'd also not object to a separation of the 8 and 16 bit implementations, but the type WIDECHAR etc. should remain at least for compatibility. We could perhaps even add some automatic conversion then between TEXT and WIDETEXT types (though this would not be in the M3 spirit :-) If we follow this way, the standard TEXTs could be replaced by the old PM3 implementation again. I'd also like to point out that much of the speed of using the old implementation comes by using the internal TextF interface, which exposes the internal TEXT representation as an array of CHAR. I'm not sure if this is really a good idea. Replacing direct array access by Text.GetChar was one of the main adaptions needed when converting all the standard packages to the CM3 compiler for the first release. I'd rather object to exposing this interface again. We could even define the standard TEXT representation as UTF8 in CM3 instead of the standard ISO Latin1 code set, which seems a bit euro- centric ;-) I'd support this idea, once we have proper UTF8 support in CM3 (which we currently haven't if I am not mistaken, see below). Quoting Dragi?a Duri? : > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. > Everything else is already transparent with regard to internal > representation. I am not sure that it is as easy as that. I'd like to see a proper update of the specification before we implement such a change. > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. No, the problem was that what I got couldn't even be compiled on FreeBSD and Solaris. We should not rely on some external GNU libraries, but we need to implement the UTF8 functions in CM3. Has anybody done that yet? > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). I'd be in favour of any improvements of the TEXT implementation and UTF8 support in CM3! > dd > > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >> Hmm, are we just victims of poor implementation? Does anyone have the >> time to improve things? It would be possible to rip out CM3 TEXT and >> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >> too. Not sure who that impacts. I'd be careful here, too. I think we need to understand the problems and the impacts of changes better before we rip out much code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From martinbishop at bellsouth.net Sat Dec 20 22:45:36 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sat, 20 Dec 2008 15:45:36 -0600 Subject: [M3devel] CM3 Source "pure"? Message-ID: <494D6780.6060701@bellsouth.net> I was looking through the CM3 sources out of boredom and noticed that there seems to be no C code, not even for runtime stuff? Are the CM3 sources "pure" in that they don't use any C code directly? (I did see some uses of M3toC and such). From hosking at cs.purdue.edu Sat Dec 20 23:05:18 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 21 Dec 2008 09:05:18 +1100 Subject: [M3devel] CM3 Source "pure"? In-Reply-To: <494D6780.6060701@bellsouth.net> References: <494D6780.6060701@bellsouth.net> Message-ID: <4D272586-0C33-4B7B-B8E6-7BD6A34C1B23@cs.purdue.edu> On 21 Dec 2008, at 08:45, Martin Bishop wrote: > I was looking through the CM3 sources out of boredom and noticed that > there seems to be no C code, not even for runtime stuff? Precisely, Modula-3 is a systems programming language just as capable as C. To that end, a goal of any language implementation is to "eat your own dog food" and implement as much of the language in itself as possible. > Are the CM3 sources "pure" in that they don't use any C code directly? > (I did see some uses of M3toC and such). There are some C sources that adapt to C library and system functions, but generally use of C is discouraged in the Modula-3 implementation. From darko at darko.org Sun Dec 21 00:08:57 2008 From: darko at darko.org (Darko) Date: Sun, 21 Dec 2008 08:08:57 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: The right way to do this, IMNSHO is to not assume any particular representation of TEXT values and create an implementation interface that allows implementations of multiple text representations, much like Rd and Wr don't make many assumptions about how data is actually stored or retrieved. This allows: - backward compatibility with the existing scheme - people's favourite representations (I personally don't think UTF-8 or UTF-16 are much chop as a single solution) - compatibility with external representation requirements (not having to convert each time you call an external API) - optimisation for particular application implementations The complication is with allowing representations to change at runtime and (automatic) conversion between representations, which I don't think adds too much complexity for the possible benefit. Also we need a proper UNICODE type that can contain only valid Unicode characters since WIDECHAR can't represent them all in one value. On 21/12/2008, at 4:27 AM, Olaf Wagner wrote: > Hi everybody, > > I haven't followed all the discussions recently, but think it may help > to comment on a few things here. > > IIRC the main reason for the current TEXT implementation was to become > compatible with Java. Java strings and M3 strings should look the same > for the embedded JVM written in M3. AFAIK this JVM has never been made > open source nor is there any intention to do it. > Still it may be a good idea to adhere to the TEXT representation if > we ever intend to combine Java and M3 in one runtime. > > I don't think the implementation of the combined 8/16 bit TEXTs is > very mature. I think that some improvements could be made. Has anybody > looked into the details of the implementation? As it is already > pointed > out on the old CM3 notes on the web pages, some things are slower > while > others are now faster. I agree that the implementation is rather > suboptimal > for some common use cases. We'd need a set of use cases and tests > agreed > upon to really compare implementations though. Just one test for cat > is > not enough IMO. This should be the first step for an improvement. > > I'd also not object to a separation of the 8 and 16 bit > implementations, > but the type WIDECHAR etc. should remain at least for > compatibility. We could perhaps even add some automatic conversion > then between TEXT and WIDETEXT types (though this would not be in the > M3 spirit :-) If we follow this way, the standard TEXTs could be > replaced by the old PM3 implementation again. > > I'd also like to point out that much of the speed of using the old > implementation comes by using the internal TextF interface, which > exposes the internal TEXT representation as an array of CHAR. I'm > not sure if this is really a good idea. Replacing direct array access > by Text.GetChar was one of the main adaptions needed when converting > all the standard packages to the CM3 compiler for the first release. > I'd rather object to exposing this interface again. > > We could even define the standard TEXT representation as UTF8 in CM3 > instead of the standard ISO Latin1 code set, which seems a bit euro- > centric ;-) I'd support this idea, once we have proper UTF8 support > in CM3 (which we currently haven't if I am not mistaken, see below). > > Quoting Dragi?a Duri? : > >> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is >> either >> slow, or produces results that make subsequent operations slow - is >> where problem is with this implementation. >> >> IMO, best solution would be to replace internal representation with >> UTF-8. For whom may be concerned with it - make some external >> widechar >> conversion routines available. >> >> That way - concat would be as fast as it can be made and other >> operations would be realistic - it is how almost everybody does their >> Unicode strings, after all. Almost everybody - excluding mobile >> industry >> and Microsoft :-), AFAIK. >> >> Compiler changes would make transparent conversion of literals. >> Everything else is already transparent with regard to internal >> representation. > > I am not sure that it is as easy as that. I'd like to see a proper > update of the specification before we implement such a change. > >> I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was >> reluctant >> to accept them as he did not believe C base routines were widespread >> enough. GNU world has no such reluctance. Everything is UTF8. > > No, the problem was that what I got couldn't even be compiled on > FreeBSD and Solaris. We should not rely on some external GNU > libraries, > but we need to implement the UTF8 functions in CM3. Has anybody done > that yet? > >> If nobody can fill this job in next few weeks, I will probably have >> some >> time this winter. No promise, of course, but list will know :-). > > I'd be in favour of any improvements of the TEXT implementation and > UTF8 support in CM3! > >> dd >> >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have >>> the >>> time to improve things? It would be possible to rip out CM3 TEXT >>> and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > > I'd be careful here, too. I think we need to understand the problems > and the impacts of changes better before we rip out much code. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From mika at async.caltech.edu Sun Dec 21 01:02:11 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 20 Dec 2008 16:02:11 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Sat, 20 Dec 2008 20:27:55 +0100." <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Olaf Wagner writes: ... >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have the >>> time to improve things? It would be possible to rip out CM3 TEXT and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > >I'd be careful here, too. I think we need to understand the problems >and the impacts of changes better before we rip out much code. > I do agree. My comments about simply putting back the PM3 code weren't meant to be taken seriously. I myself have always disliked this "internationalization" business. I remember my father used to have to prefix prices in Telexes to England with an o-with-umlaut (the o-with-umlaut on his Swedish Siemens printed as a lira on a British machine). And there's always been something vaguely Socialistic about it: I don't mean the fact that I had to use A-umlaut and Scandinavian-A-with-circle for square brackets on the C-64, but that on the very first computer I ever used, the Swedish Luxor ABC-80 (Zilog Z80 based), the Swedish standard of the time called for the dollar sign to be replaced by a small "sun symbol" pronounced "sol" in Swedish. BASIC strings were A-sol and B-sol, etc. I do not believe the "sol" is any more useful than the "dollar" in Sweden (probably less, especially in the winter); I have a feeling that Swedish standards bodies at the time were simply trying to show their sympathies for the money-less regime of the Khmer Rouge. But that being said, I realize there are people out there who might want non-ASCII in their strings. I think if the CM3 implementation does that well it might be worth fixing... not sure that UTF encoding is any simpler, overall...? A peek at Text.Equal sheds some light on the problem: PROCEDURE Equal (t, u: T): BOOLEAN = VAR info_t, info_u: Info; BEGIN t.get_info (info_t); u.get_info (info_u); IF (info_t.length # info_u.length) THEN RETURN FALSE; END; IF (info_t.length = 0) THEN RETURN TRUE; END; IF (info_t.start = NIL) OR (info_u.start = NIL) OR (info_t.wide # info_u.wide) THEN RETURN EqualBuf (t, u, info_t.length); ELSIF NOT info_t.wide THEN RETURN String8.Equal (info_t.start, info_u.start, info_t.length); ELSE RETURN String16.Equal (info_t.start, info_u.start, info_t.length); END; END Equal; Under CM3, Text.Cat/RTHooks.Concat (&) returns a special subtype of string, viz., TextCat.T, which has the following for get_info: PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; The NIL for info.start is what pushes the code into EqualBuf, which compares the strings character by character, but asks for each character via the get_wide_chars method, which calls itself recursively for each concatenated component. info.start = NIL is, in short, a flag that means "non-standard TEXT, use heavily object-oriented character-by-character routines". It seems to me this implementation could be sped up a great deal by doing some simple micro-optimizations. Let me ask a very simple question... is it possible to sub-type TEXT in CM3? (Is such an interface exported?) If not, it would be quite easy to de-object-orient this code and get most of the PM3 performance back, I think (we'd have to flatten the strings a bit too). If the code can be subtyped by clients outside of the text package it would be necessary to sort out what types of overrides should be supported, and what should happen if someone, for instance, subtypes a TextCat.T to have a different method for getting characters... I will try to get some time to look at this myself over the holidays. Note to self and others: m3gdb is not completely useful here. If you look at the structure of a TextCat.T, you see that fields a and b are TEXTs, but m3gdb doesn't really let you look inside them, since, well, they are TEXTs and printed as such. (You don't get to see the whole gory mess of pointers.) Is there a way to turn off this (normally very helpful) m3gdb feature? Mika From hendrik at topoi.pooq.com Sun Dec 21 21:40:03 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Sun, 21 Dec 2008 15:40:03 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081221204002.GA26711@topoi.pooq.com> On Sat, Dec 20, 2008 at 10:10:45AM +0100, Dragi?a Duri? wrote: > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. UTF-8 is indeed designed so that a lot of operations can be oerformed directly on the bytes of UTF-8. For example, if you sort UTF-8 strings based on the 8-bit unsigned values of their bytes, you get the same order as if you sorted them based on the integer values of their Unicode characters. As for delimiting strings based on character in them, such as single-byte spaces, you can just process it as a byte stream. If you want to find a character that's, say, three UTF8 bytes, you can do ordinary byte-string search for a three-byte string -- UTF-8 has self-delimiting characters. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. I worked on a C compiler long ago, and read the standard in great detail to determine that quoted strings contained characters, not bytes. As a result, the compiler we wrked on had a bug when it came to handling Korean characters when working in a Korean environ -- one of the Korean two-byte characters happened to contain a null byte, and the compiler treated it internally as end-of-string. Of course, we fixed, that, using locale-dependent character-parsing routines. Now Unicode doesn't even have that problem -- the only character containing a null byte is the null character, and it's a single null byte. (Java messes this up by using a different encoding for the null character, so that they can treat their compactly-encoded strings as being null-terminated. But they don't claim to used UTF-8 either, though they do almost use it. The encoding of the null character they use is such that almost any UTF-8 DEcoder will decode it as being a NULL character, unless it is specifically designed to check for it as an error condition. Just look ar the high-oprder bits of the bytes and do your shifts and you'll be OK.) I routinely use UTF-8 as the normal character code in all the software I write, and I almost never have an occasion when library-provided encoding and decoding functions are of much use. If I care much about the syntax, I'm using a parser anyway, and a parser can just as easlily parse bytes with the high bit set as bytes without. -- hendrik > Everything else is already transparent with regard to internal > representation. > > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. > > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). What's the job in question? Code that handles UTF-8? New data structures just for UTF-8 strings? Or code that just processes ordinary strings and does special processing when indicated on bytes with the high bit set? -- hendrik From dragisha at m3w.org Sun Dec 21 11:33:18 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 21 Dec 2008 11:33:18 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <494D3B89.3020802@gmx.de> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> Message-ID: <1229855598.18256.168.camel@faramir.m3w.org> Most important argument for UTF-8 - IMO - would be it's completeness. 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit storage magic instead? Accepting 2 byte would be same mistake once over. Once Latin1 was whole world. Is it 2byte now? As for "length" argument - it is cacheable data. As our TEXT is object type, we can do such things. As for "substr", it is sound counter argument. Internal optimizations are possible, of course. For lengthy strings, we can implement it, and efficiently. Once internal representation of TEXT was possible to pass to C without conversion. With UTF8 - it's possible again. Java is not world around us - it's C. And again - it's not it's storage efficiency that coounts - it's completeness. On Sat, 2008-12-20 at 19:38 +0100, Roland Illig wrote: > Dragi?a Duri? schrieb: > > IMO, best solution would be to replace internal representation with > > UTF-8. For whom may be concerned with it - make some external widechar > > conversion routines available. > > > > That way - concat would be as fast as it can be made and other > > operations would be realistic - it is how almost everybody does their > > Unicode strings, after all. Almost everybody - excluding mobile industry > > and Microsoft :-), AFAIK. > > I object. UTF-8 is a good external representation format, but in memory, > it is far more efficient to let all characters of a string have the same > size. Java, for example, has defined a character to be "what fits into > 16 bits". That's good enough in many situations. One of the situations > it fails is when you want to represent Domino Tiles or Mahjong Tiles as > characters. > > To give you some arguments against UTF-8 as in-memory representation: > > * How fast can you calculate the length of a string? (With length I mean > the number of characters, not the memory size needed for storing them.) > > * Where does substring(5) start? Where does substring(10000) start, and > how long does it take to find out? > > An argument for the UTF-8 representation is that it saves memory -- one > byte for every ASCII character. Is this argument still as important as > it has been 40 years ago? > > Roland -- Dragi?a Duri? From stsp at elego.de Sun Dec 21 16:40:15 2008 From: stsp at elego.de (Stefan Sperling) Date: Sun, 21 Dec 2008 15:40:15 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <20081221154015.GA6543@jack.stsp.name> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: > The right way to do this, IMNSHO is to not assume any particular > representation of TEXT values and create an implementation interface > that allows implementations of multiple text representations, much like > Rd and Wr don't make many assumptions about how data is actually stored > or retrieved. Such an interface may be needed for UTF-8 alone already, anyway, because within UTF-8 there is in some cases more than one way to store what amounts to the same data to a human user. In Subversion, from the beginning everyone agreed that the internal encoding for all strings would be UTF-8. Most Subversion APIs expect data in UTF-8. Strings (e.g. filenames) in the repository are stored in UTF-8, etc. Great! Will work in all countries! Right? Yes, but not on all operating systems if you're not careful! It did not occur to anyone at the time that there are characters which in UTF-8 have more than one representation (codepoints) in a byte stream. For example, an u with umlaut can be encoded as two bytes or a single byte: 2 bytes: [u | the previous character has an umlaut ] This is called "normal form decomposed". 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) This is called "normal form composed". If you want to be portable, as CM3 and Subversion both try to be, you have to consider that some operating systems may return your filenames in a different encoding then you stored it in: -------- Accepts Gives back MacOS X * NFD(*) Linux * Windows * Others ? ? *) There are some remarks to be made regarding full or partial NFD here, but the essential thing is: If you send in NFC, don't expect it back! -------- quoted from: http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames which is worth a read for more details if you're interested. In Subversion, this is a real problem for Mac users, because two filenames which only differ in their NFC/NFD encoding look exactly the same to the user (an u umlaut is printed), while the byte streams do not match ("We're sorry, but your file x does not exist in the repository!", where x looks just like a file that is clearly visible in the repository listing :) Subversion's problem now is that there are repositories out there using filenames in either NFC, NFD, or mixed, and there is no good way to reconcile the mess while staying backwards compatible with existing clients, servers, working copies and repositories. So Mac users are told to only use ASCII characters in their filenames to prevent the problem (many users, especially users who are not programmers, who store their photos or their entire home directory or whatever in Subversion, are not happy about this). This problem may not matter as much in case of CM3, but anyone implementing UTF-8 support for CM3 should be aware of this issue and not repeat the mistake the Subversion developers made at the time! With UTF-8, do not rely on a filename to retain its encoding as you passed it to the OS when requesting the filename from the OS again. CM3 should pick either NFD or NFC as internal UTF-8 encoding, for filenames only, or for all strings, whichever makes more sense. And then stick to it, converting input/output as needed. Abstracting this problem away using a nice interface would probably be the cleanest solution. Stefan From jay.krell at cornell.edu Sun Dec 21 18:51:14 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:51:14 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: This is an issue with Unicode, not UTF8, right? ie: a 16 or 20 or 32 bit encoding has the same problem, right? search the web for "unicode precomposed": http://en.wikipedia.org/wiki/Precomposed_character http://wikisource.org/wiki/Unicode_precomposed_characters or "unicode precomposed apple": http://developer.apple.com/jp/qa/qa2001/qa1235.html http://developer.apple.com/qa/qa2001/qa1235.html "When working within Mac OS you will find yourself using a mixture of precomposed and decomposed Unicode. For example, HFS Plus converts all file names to decomposed Unicode, while Macintosh keyboards generally produce precomposed Unicode. This isn't a problem as long as you use system-provided APIs to process text. Apple's APIs correctly handle both precomposed and decomposed Unicode. However, you may need to convert to precomposed Unicode when you interact with other platforms. For example, the following are all valid reasons why you might want to convert to precomposed Unicode. If you implement a network protocol which is defined to use precomposed Unicode. When creating a cross-platform file (or volume) whose specification dictates precomposed Unicode. If you incorporate a large body of cross-platform code into your application, where that code is expecting precomposed Unicode. " http://www.unicode.org/unicode/reports/tr15/index.html I need to read these.. - Jay> Date: Sun, 21 Dec 2008 15:40:15 +0000> From: stsp at elego.de> To: darko at darko.org> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] This disgusting TEXT business> > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote:> > The right way to do this, IMNSHO is to not assume any particular > > representation of TEXT values and create an implementation interface > > that allows implementations of multiple text representations, much like > > Rd and Wr don't make many assumptions about how data is actually stored > > or retrieved.> > Such an interface may be needed for UTF-8 alone already, anyway,> because within UTF-8 there is in some cases more than one way> to store what amounts to the same data to a human user.> > In Subversion, from the beginning everyone agreed that the internal> encoding for all strings would be UTF-8. Most Subversion APIs expect> data in UTF-8. Strings (e.g. filenames) in the repository are stored> in UTF-8, etc. Great! Will work in all countries! Right?> > Yes, but not on all operating systems if you're not careful!> It did not occur to anyone at the time that there are characters> which in UTF-8 have more than one representation (codepoints) in a> byte stream. For example, an u with umlaut can be encoded as two> bytes or a single byte:> > 2 bytes: [u | the previous character has an umlaut ]> This is called "normal form decomposed".> > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :)> This is called "normal form composed".> > If you want to be portable, as CM3 and Subversion both try to be,> you have to consider that some operating systems may return your> filenames in a different encoding then you stored it in:> > --------> Accepts Gives back> MacOS X * NFD(*)> Linux * > Windows * > Others ? ?> > > *) There are some remarks to be made regarding full or partial> NFD here, but the essential thing is: If you send in NFC, don't> expect it back!> -------- quoted from:> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames> which is worth a read for more details if you're interested.> > In Subversion, this is a real problem for Mac users, because> two filenames which only differ in their NFC/NFD encoding> look exactly the same to the user (an u umlaut is printed),> while the byte streams do not match ("We're sorry, but your> file x does not exist in the repository!", where x looks just> like a file that is clearly visible in the repository listing :)> > Subversion's problem now is that there are repositories out> there using filenames in either NFC, NFD, or mixed, and there> is no good way to reconcile the mess while staying backwards> compatible with existing clients, servers, working copies and> repositories. So Mac users are told to only use ASCII characters> in their filenames to prevent the problem (many users, especially> users who are not programmers, who store their photos or their> entire home directory or whatever in Subversion, are not happy> about this).> > This problem may not matter as much in case of CM3, but anyone> implementing UTF-8 support for CM3 should be aware of this issue> and not repeat the mistake the Subversion developers made at the> time! With UTF-8, do not rely on a filename to retain its encoding> as you passed it to the OS when requesting the filename from the> OS again.> > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for> filenames only, or for all strings, whichever makes more sense.> And then stick to it, converting input/output as needed.> > Abstracting this problem away using a nice interface would probably> be the cleanest solution.> > Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 21 18:57:00 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:57:00 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <465668.39592.qm@web28601.mail.ukl.yahoo.com> References: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Message-ID: The real world is unsafe.Safe areas are created by unsafe code, which must be, to a certain extent, bug free, in order to provide as safe as advertised. I don't believe what I added is any more untrusted/unsafe than what was there before.Well, maybe in a minor way. m3core has "lots" of "external" functions, which are called by "lots" of functions at higher levels..ok, maybe not "lots", but if you look at the m3core/src/unix directories that I have worked on -- cygwin, openbsd, amd64_linux, linux-common -- this is a fairly minimal set of stuff that is being used by preexisting code. A LITTLE bit I added beyond what was there. If you look at all the other directories, you'll find lots more external stuff, most of which is not used within the CM3 codebase (but could be used by folks outside). It is true I may have introduced external where it wasn't previously, but I don't think it matters.It is prevalent one level down. Moving some of it one level up is ok I believe. Non-Posix does potentially have a place in .c files.It depends.Basically, as I see things, a small amount of #ifdef is ok.A lot of #ifdef becomes hard to read, and at some point, you split the files into separate files, "no" #ifdefs -- or conceptually each file can be viewed as wrapped in an #ifdef or other platform detection.You need #ifdef in any C that uses 64 bit integers, for example. Current Microsoft compilers do support "long long" but __int64 supporg goes way back, and __int64 is a better name anyway. What is the 128 bit integer type to be called -- "__int128" or "long long long". The names "char", "short", "int", "long", "long long" are mostly just funny mistaken names. The right names to provide are explicitly sized, int8, uint8, int16, uint16, int32, uint32, int64, uint64, and pointer sized types such as "size_t" and "diff_t". Modula-3 I think is reasonable here, since it's primary integral type is pointer sized, and it has "bits x for ...", though the other types probably ought to be built in and blessed somewhere, not just in the Csupport library section. Not sure what "Posix" means, since it might incorporate by-reference ANSI C. - Jay Date: Sat, 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3devel at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Dear all:I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have.However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not claim any more they are safe so that "fix" is not feasible.Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file.Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ), that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation.There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that?Thanks in advance--- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is ok now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Dec 22 15:20:12 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 09:20:12 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229855598.18256.168.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> <1229855598.18256.168.camel@faramir.m3w.org> Message-ID: <20081222142011.GB28224@topoi.pooq.com> On Sun, Dec 21, 2008 at 11:33:18AM +0100, Dragi?a Duri? wrote: > Most important argument for UTF-8 - IMO - would be it's completeness. > 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some > obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit > storage magic instead? Well, 20 bits would certianly fit inside 16 bytes, :-) -- hendrik From darko at darko.org Sun Dec 21 22:07:21 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 06:07:21 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> The way to handle this would be to require an iterate function on each text representation which successively returned the next character in a string as a UNICHAR with defined characteristics. That would allow for comparison and transfer of strings. The way I see it, TEXT should be revealed to be a subtype of the "TextObj" object that has methods that basically mirror the Text interface with some additions. The existing text representation code is adapted to this object (fairly simple) and new representations subclass that object. A new function is added to text interface that allows the user to specify the default representation used when creating new strings. This would be the typecode of the object to allocate, required to be a subtype of TextObj of course. Some TextObj methods would be required to be implemented by the representation but most could be handled by the ancestor. Many of the methods could be implemented by the encoding for more efficient handling, for instance when the text being compared or concatenated is the same representation. UNICHAR = ; TextObj = OBJECT METHODS length(t: T): CARDINAL; (* required to be implemented by the encoding *) empty(t: T): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(t: T): BOOLEAN; (* required - meaning containing anything other than CHAR values *) next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index otherwise meaningless and private. Seek allows skipping forward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(t, u: T): BOOLEAN; compare(t1, t2: T): [-1..1]; cat(t, u: T): T; sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(t: T): Word.T; getChar(t: T; i: CARDINAL): CHAR; getWideChar(t: T; i: CARDINAL): WIDECHAR; getUniChar(t: T; i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWIdeChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(t: T; c: CHAR; start := 0): INTEGER; findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; Additionally the Text and a couple of other interfaces (eg Rd, Wr) would need to be expanded to handle UniChar. On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >> The right way to do this, IMNSHO is to not assume any particular >> representation of TEXT values and create an implementation interface >> that allows implementations of multiple text representations, much >> like >> Rd and Wr don't make many assumptions about how data is actually >> stored >> or retrieved. > > Such an interface may be needed for UTF-8 alone already, anyway, > because within UTF-8 there is in some cases more than one way > to store what amounts to the same data to a human user. > > In Subversion, from the beginning everyone agreed that the internal > encoding for all strings would be UTF-8. Most Subversion APIs expect > data in UTF-8. Strings (e.g. filenames) in the repository are stored > in UTF-8, etc. Great! Will work in all countries! Right? > > Yes, but not on all operating systems if you're not careful! > It did not occur to anyone at the time that there are characters > which in UTF-8 have more than one representation (codepoints) in a > byte stream. For example, an u with umlaut can be encoded as two > bytes or a single byte: > > 2 bytes: [u | the previous character has an umlaut ] > This is called "normal form decomposed". > > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) > This is called "normal form composed". > > If you want to be portable, as CM3 and Subversion both try to be, > you have to consider that some operating systems may return your > filenames in a different encoding then you stored it in: > > -------- > Accepts Gives back > MacOS X * NFD(*) > Linux * > Windows * > Others ? ? > > > *) There are some remarks to be made regarding full or partial > NFD here, but the essential thing is: If you send in NFC, don't > expect it back! > -------- quoted from: > http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames > which is worth a read for more details if you're interested. > > In Subversion, this is a real problem for Mac users, because > two filenames which only differ in their NFC/NFD encoding > look exactly the same to the user (an u umlaut is printed), > while the byte streams do not match ("We're sorry, but your > file x does not exist in the repository!", where x looks just > like a file that is clearly visible in the repository listing :) > > Subversion's problem now is that there are repositories out > there using filenames in either NFC, NFD, or mixed, and there > is no good way to reconcile the mess while staying backwards > compatible with existing clients, servers, working copies and > repositories. So Mac users are told to only use ASCII characters > in their filenames to prevent the problem (many users, especially > users who are not programmers, who store their photos or their > entire home directory or whatever in Subversion, are not happy > about this). > > This problem may not matter as much in case of CM3, but anyone > implementing UTF-8 support for CM3 should be aware of this issue > and not repeat the mistake the Subversion developers made at the > time! With UTF-8, do not rely on a filename to retain its encoding > as you passed it to the OS when requesting the filename from the > OS again. > > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for > filenames only, or for all strings, whichever makes more sense. > And then stick to it, converting input/output as needed. > > Abstracting this problem away using a nice interface would probably > be the cleanest solution. > > Stefan From hendrik at topoi.pooq.com Mon Dec 22 16:54:38 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 10:54:38 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <20081222155438.GA28375@topoi.pooq.com> On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: > The way to handle this would be to require an iterate function on each > text representation which successively returned the next character in > a string as a UNICHAR with defined characteristics. That would allow > for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some TextObj > methods would be required to be implemented by the representation but > most could be handled by the ancestor. Many of the methods could be > implemented by the encoding for more efficient handling, for instance > when the text being compared or concatenated is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) How is T related to TextObj? Does this return the number of characters of the number of bytes occupied by the stinrg? > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) Does this require a scan of the string to determine whether any wide characters are actually present, or does it just indicate whether the data representation used in this string is capable of handling wide characters? > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the next > logical character in char and returns true, or false if the char > doesn't exist. Index otherwise meaningless and private. Seek allows > skipping forward a number of characters. *) Do we need a colon after the second VAR? This seems intended for an implementation where index is a byte offset. Is it possible to copy index so as to start a new interatin where at a saved point? > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) What stack? > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) length in bytes or characters? > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; I'd really like functions that provide access to the underlying UTF8 encoding on a byte-by-byte basis. Often that's the most efficient way, and the simplest, to process UTF8-encoded data, and UTF8 had been designed with this in mind. > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > while we are at it, let's include byte access as well. Treating bytes as integers between 0 and 255 would work. -- hendrik Note: when I'm processing UTF8 data, I'm often parsing it and inserting other tokens (which are not UTF8 characters) into the data stream as well. It is convenient to use negative numbers for this, clearly distinguished from Unicode codepints, which are positive. So making characters availabla as positive integers would work cleanly with this. From darko at darko.org Mon Dec 22 01:07:39 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:07:39 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081222155438.GA28375@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> Message-ID: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> On 23/12/2008, at 12:54 AM, hendrik at topoi.pooq.com wrote: > On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: >> The way to handle this would be to require an iterate function on >> each >> text representation which successively returned the next character in >> a string as a UNICHAR with defined characteristics. That would allow >> for comparison and transfer of strings. >> >> The way I see it, TEXT should be revealed to be a subtype of the >> "TextObj" object that has methods that basically mirror the Text >> interface with some additions. The existing text representation code >> is adapted to this object (fairly simple) and new representations >> subclass that object. A new function is added to text interface that >> allows the user to specify the default representation used when >> creating new strings. This would be the typecode of the object to >> allocate, required to be a subtype of TextObj of course. Some TextObj >> methods would be required to be implemented by the representation but >> most could be handled by the ancestor. Many of the methods could be >> implemented by the encoding for more efficient handling, for instance >> when the text being compared or concatenated is the same >> representation. >> >> UNICHAR = ; >> >> TextObj = OBJECT METHODS >> >> length(t: T): CARDINAL; (* required to be implemented by the >> encoding *) > > How is T related to TextObj? Does this return the number of > characters > of the number of bytes occupied by the stinrg? Sorry, T = TEXT. It returns the logical number of characters according to some definition, which I imagine would be Unicode NFC (Normalised Form, Canonical Composition). > > >> empty(t: T): BOOLEAN; (* required to be implemented by the encoding >> *) >> hasWideChars(t: T): BOOLEAN; (* required - meaning containing >> anything other than CHAR values *) > > Does this require a scan of the string to determine whether any wide > characters are actually present, or does it just indicate whether the > data representation used in this string is capable of handling wide > characters? No to both. It doesn't require a scan, you can cache this flag. It's so you can possibly be more efficient when you know that you only have to deal with CHAR values. > > >> next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): >> BOOLEAN; (* required - start iterating with index=0, returns the next >> logical character in char and returns true, or false if the char >> doesn't exist. Index otherwise meaningless and private. Seek allows >> skipping forward a number of characters. *) > > Do we need a colon after the second VAR? No, it's a typo. > This seems intended for an implementation where index is a byte > offset. > Is it possible to copy index so as to start a new interatin where at a > saved point? Yes. > > >> getData(): ADDRESS; (* required - get the raw data, only valid >> while on the stack *) > > What stack? The execution stack. It might be a REFANY so would change if it didn't appear in the stack. > > >> setData(adr: ADDRESS; length: CARDINAL); (* required - set the data >> and length of the encoded data *) > > length in bytes or characters? Bytes, it's the length of the raw data and has no logical interpretation. > > >> >> (* the remaining methods are optional for the encoding >> implementation *) >> equal(t, u: T): BOOLEAN; >> compare(t1, t2: T): [-1..1]; >> cat(t, u: T): T; >> sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; >> hash(t: T): Word.T; >> getChar(t: T; i: CARDINAL): CHAR; >> getWideChar(t: T; i: CARDINAL): WIDECHAR; >> getUniChar(t: T; i: CARDINAL): UNICHAR; > > I'd really like functions that provide access to the underlying UTF8 > encoding on a byte-by-byte basis. Often that's the most efficient > way, > and the simplest, to process UTF8-encoded data, and UTF8 had been > designed with this in mind. If you know the text is encoded as UTF-8 then you can get the raw data as above. > > >> setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); >> setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); >> setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); >> fromChar(ch: CHAR): T; >> fromWIdeChar(ch: WIDECHAR): T; >> fromUniChar(ch: UNICHAR): T; >> fromChars(READONLY a: ARRAY OF CHAR): T; >> fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; >> fromUniChars(READONLY a: ARRAY OF UNICHAR): T; >> findChar(t: T; c: CHAR; start := 0): INTEGER; >> findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; >> findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; >> findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; >> findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; >> findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; >> END; >> >> Additionally the Text and a couple of other interfaces (eg Rd, Wr) >> would need to be expanded to handle UniChar. >> > > while we are at it, let's include byte access as well. Treating bytes > as integers between 0 and 255 would work. Can't see the point since you can ORD a char value to get the same. > > > -- hendrik > > Note: when I'm processing UTF8 data, I'm often parsing it and > inserting > other tokens (which are not UTF8 characters) into the data stream as > well. It is convenient to use negative numbers for this, clearly > distinguished from Unicode codepints, which are positive. So making > characters availabla as positive integers would work cleanly with > this. > > From darko at darko.org Mon Dec 22 01:14:16 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:14:16 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> A couple of corrections: - T = TEXT and TextObj <: T so it should read TextObj = T OBJECT - the cat method is required for the encoding implementation (helps to actually be able to create strings) - the CARDINAL in the next method should be INTEGER to enable searching backwards (findCharR) and should probably be named 'iter'. On 22/12/2008, at 6:07 AM, Darko wrote: > The way to handle this would be to require an iterate function on > each text representation which successively returned the next > character in a string as a UNICHAR with defined characteristics. > That would allow for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some > TextObj methods would be required to be implemented by the > representation but most could be handled by the ancestor. Many of > the methods could be implemented by the encoding for more efficient > handling, for instance when the text being compared or concatenated > is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the > next logical character in char and returns true, or false if the > char doesn't exist. Index otherwise meaningless and private. Seek > allows skipping forward a number of characters. *) > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > > > > On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > >> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >>> The right way to do this, IMNSHO is to not assume any particular >>> representation of TEXT values and create an implementation interface >>> that allows implementations of multiple text representations, much >>> like >>> Rd and Wr don't make many assumptions about how data is actually >>> stored >>> or retrieved. >> >> Such an interface may be needed for UTF-8 alone already, anyway, >> because within UTF-8 there is in some cases more than one way >> to store what amounts to the same data to a human user. >> >> In Subversion, from the beginning everyone agreed that the internal >> encoding for all strings would be UTF-8. Most Subversion APIs expect >> data in UTF-8. Strings (e.g. filenames) in the repository are stored >> in UTF-8, etc. Great! Will work in all countries! Right? >> >> Yes, but not on all operating systems if you're not careful! >> It did not occur to anyone at the time that there are characters >> which in UTF-8 have more than one representation (codepoints) in a >> byte stream. For example, an u with umlaut can be encoded as two >> bytes or a single byte: >> >> 2 bytes: [u | the previous character has an umlaut ] >> This is called "normal form decomposed". >> >> 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) >> This is called "normal form composed". >> >> If you want to be portable, as CM3 and Subversion both try to be, >> you have to consider that some operating systems may return your >> filenames in a different encoding then you stored it in: >> >> -------- >> Accepts Gives back >> MacOS X * NFD(*) >> Linux * >> Windows * >> Others ? ? >> >> >> *) There are some remarks to be made regarding full or partial >> NFD here, but the essential thing is: If you send in NFC, don't >> expect it back! >> -------- quoted from: >> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames >> which is worth a read for more details if you're interested. >> >> In Subversion, this is a real problem for Mac users, because >> two filenames which only differ in their NFC/NFD encoding >> look exactly the same to the user (an u umlaut is printed), >> while the byte streams do not match ("We're sorry, but your >> file x does not exist in the repository!", where x looks just >> like a file that is clearly visible in the repository listing :) >> >> Subversion's problem now is that there are repositories out >> there using filenames in either NFC, NFD, or mixed, and there >> is no good way to reconcile the mess while staying backwards >> compatible with existing clients, servers, working copies and >> repositories. So Mac users are told to only use ASCII characters >> in their filenames to prevent the problem (many users, especially >> users who are not programmers, who store their photos or their >> entire home directory or whatever in Subversion, are not happy >> about this). >> >> This problem may not matter as much in case of CM3, but anyone >> implementing UTF-8 support for CM3 should be aware of this issue >> and not repeat the mistake the Subversion developers made at the >> time! With UTF-8, do not rely on a filename to retain its encoding >> as you passed it to the OS when requesting the filename from the >> OS again. >> >> CM3 should pick either NFD or NFC as internal UTF-8 encoding, for >> filenames only, or for all strings, whichever makes more sense. >> And then stick to it, converting input/output as needed. >> >> Abstracting this problem away using a nice interface would probably >> be the cleanest solution. >> >> Stefan > From hendrik at topoi.pooq.com Mon Dec 22 21:10:36 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 15:10:36 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> Message-ID: <20081222201036.GC28728@topoi.pooq.com> On Mon, Dec 22, 2008 at 09:07:39AM +0900, Darko wrote: > > > > > >> getData(): ADDRESS; (* required - get the raw data, only valid > >>while on the stack *) > > > >What stack? > > The execution stack. It might be a REFANY so would change if it didn't > appear in the stack. I must be misunderstanding something; this seems perfectly obvious to you, but opaque to me. > >> > > > >while we are at it, let's include byte access as well. Treating bytes > >as integers between 0 and 255 would work. > > Can't see the point since you can ORD a char value to get the same. Not unless it's *specified* that the raw data structure *is* a bunch of CHARs, and that CHARs are single bytes. This whole discussion is occasioned by the discovery that characters are no longer single bytes, but strings of bytes. -- hendrik From darko at darko.org Mon Dec 22 03:52:38 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 11:52:38 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> Message-ID: <5A505AA9-97B5-4B20-878C-0FDC5359B8A7@darko.org> Ah the perils of copy and paste early in the morning. Here is a corrected version without extraneous method parameters. UNICHAR = ; T = TEXT; T <: TextObj; TextObj = OBJECT METHODS length(): CARDINAL; (* required to be implemented by the encoding *) empty(): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(): BOOLEAN; (* required - meaning containing anything other than CHAR values *) cat(u: T): T; (* required to be implemented by the encoding *) iter(VAR index: INTEGER; VAR char: UNICHAR; seek: INTEGER := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index is otherwise meaningless and private. Seek allows skipping forward or backward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(u: T): BOOLEAN; compare(u: T): [-1..1]; sub(start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(): Word.T; getChar(i: CARDINAL): CHAR; getWideChar(i: CARDINAL): WIDECHAR; getUniChar(i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWideChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWideChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(c: CHAR; start := 0): INTEGER; findWideChar(c: WIDECHAR; start := 0): INTEGER; findUniChar(c: UNICHAR; start := 0): INTEGER; findCharR(c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; PROCEDURE SetTextEnc(tc: INTEGER); (* set the encoding object to use from its typecode, must be subtype of TextObj *) PROCEDURE GetTextEnc(): INTEGER: (* returns the typecode of the current encoding object in use *) From hosking at cs.purdue.edu Mon Dec 22 21:56:50 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 23 Dec 2008 07:56:50 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: > From: Tony Hosking > Date: 23 December 2008 00:04:20 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:30:06 -- checkout in progress. > [start checkout 2008.12.22 06:30:10] > cd /tmp/build-cm3-20081222-063006-5jaOwe/build > cvs return value: 0 > [end checkout 2008.12.22 06:49:53] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 06:49:55 -- compile in progress. > [start compile 2008.12.22 06:49:55] > compile return value: 0 > [end compile 2008.12.22 06:55:09] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:57:16 -- checkout in progress. > [start checkout 2008.12.22 06:57:18] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > cvs return value: 0 > [end checkout 2008.12.22 07:16:39] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 07:16:41 -- compile in progress. > [start compile 2008.12.22 07:16:41] > compile return value: 0 > [end compile 2008.12.22 08:02:05] > COMPILE_RETURN = 0 > 2008.12.22 08:02:22 -- tests in progress. > [start run-tests 2008.12.22 08:02:22] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > [end run-tests 2008.12.22 08:02:22] > TESTS_RETURN = 0 > 2008.12.22 08:02:22 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 04:06:37 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Mon, 22 Dec 2008 21:06:37 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> References: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Message-ID: <495055BD.2010009@wichita.edu> Mika Nystrom wrote: > Note to self and others: m3gdb is not completely useful here. If > you look at the structure of a TextCat.T, you see that fields a and > b are TEXTs, but m3gdb doesn't really let you look inside them, > since, well, they are TEXTs and printed as such. (You don't > get to see the whole gory mess of pointers.) Is there a way to > turn off this (normally very helpful) m3gdb feature? Yes. Perhaps a couple of years ago, I added a /k format letter to m3gdb for printing TEXT values showing the internal object, rather than the characters in quotes. At the time, I only announced this in the CVS checkin logs (which show up in the m3commit mailing list), but recently, I added a good-sized bit of collected documentation for m3gdb, which describes this in more detail. From: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html#id2517604 ----------------------------------------------------------------------------- Normally, m3gdb displays TEXT values in Modula-3 lexical syntax, with double quotes, escape sequences, and, if appropriate, a leading 'W'. However, the /k option in a print command will cause it instead to display the TEXT value's internal data structure. For PM3 et. al., this is a traced reference to an open array of characters. For CM3, this will be one of the several object subtypes of TEXT. m3gdb properly recognizes and displays values of CM3 type TextLiteral.T. Here, it takes the length of the text from the appropriate field, instead of from the declared type, which contains a fixed array whose length is amost always far too long. Even without specifying the /k option, if you happen to know what the internal representation is, you can apply appropriate operators to it, e.g., dereferencing, field selection, subscripting, or even method calls. ----------------------------------------------------------------------------- Examples: (m3gdb) p T6 $15 = (*16_b740e1dc*) "TextLiteralabcdefghij" (m3gdb) p/k T6 $16 = (*16_b740e1dc*) OBJECT a = 16_0804a670; b = 16_b740e0c8; a_len = 11; b_len = 10; a_or_b_wide = FALSE; END (m3gdb) p/k T6.a $17 = (*16_0804a670*) (*TextLiteral.T*) cnt = 11; buf = "TextLiteral" END (m3gdb) p/k T6.b $18 = (*16_b740e0c8*) OBJECT len = 10; contents = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', '\000', '\000', '\000', '\000', '\000', '\000'}; END ----------------------------------------------------------------------------- It would be nicer if m3gdb also always printed the name of the type of the object, but I haven't figured out what to do here in general, because an object type can have no name, or more than one name. But I see I am now printing (*TextLiteral.T*) as a special case, so the other CM3 subtypes of TEXT could be handled the same way. 'Nother item for my long m3gdb to-do list. Rodney Bates > > Mika > > From jay.krell at cornell.edu Tue Dec 23 13:53:26 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 12:53:26 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Drat. I fixed a likely problem (broken on other platforms) and I think "upgrade" succeeded, but then I messed things up -- no working gcc. It looks like I might not be home now till just after New Years due to unforseen circumstances. (very inclement weather, stuck elsewhere) Can you get a more specific error message? If so, I can checkin a guess late in the week. Otherwise, can it wait? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Tue, 23 Dec 2008 07:56:50 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: From: Tony Hosking Date: 23 December 2008 00:04:20 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:30:06 -- checkout in progress.[start checkout 2008.12.22 06:30:10]cd /tmp/build-cm3-20081222-063006-5jaOwe/buildcvs return value: 0[end checkout 2008.12.22 06:49:53]CHECKOUT_RETURN = 0--2008.12.22 06:49:55 -- compile in progress.[start compile 2008.12.22 06:49:55]compile return value: 0[end compile 2008.12.22 06:55:09]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081222-063006-5jaOwe ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:57:16 -- checkout in progress.[start checkout 2008.12.22 06:57:18]cd /tmp/build-cm3-20081222-065716-Oha4Uk/buildcvs return value: 0[end checkout 2008.12.22 07:16:39]CHECKOUT_RETURN = 0--2008.12.22 07:16:41 -- compile in progress.[start compile 2008.12.22 07:16:41]compile return value: 0[end compile 2008.12.22 08:02:05]COMPILE_RETURN = 02008.12.22 08:02:22 -- tests in progress.[start run-tests 2008.12.22 08:02:22]cd /tmp/build-cm3-20081222-065716-Oha4Uk/build[end run-tests 2008.12.22 08:02:22]TESTS_RETURN = 02008.12.22 08:02:22 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Dec 23 13:59:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 23 Dec 2008 13:59:00 +0100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Quoting Tony Hosking : > Jay, things are still failing on my SOLgnu installation after your > changes. Did you ever try it on yours? According to tinderbox, all release builds fail due to undefined quake procedures: /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? " m3cc 1196 === package m3-sys/m3cc === 1197 +++ cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? +++ 1198 --- building in SOLgnu --- 1199 1200 NEXT "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: fs_exists 1201 fs_exists cannot be used unless we provide an official release before, because it has not been officially released yet. Some lastok-builds fail for other reasons. Has anybody time to fix these? Alternatively, we could back out the offending changes... > Happy holidays to all! Happy Christmas to all M3 users from me, too! Olaf > Begin forwarded message: > >> From: Tony Hosking >> Date: 23 December 2008 00:04:20 GMT+11:00 >> To: hosking at cs.purdue.edu >> Subject: Output from "cron" command >> >> Your "cron" job on niagara.cs.purdue.edu >> $HOME/cm3/scripts/regression/cron.sh >> >> produced the following output: >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara release-build" >> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:30:06 -- checkout in progress. >> [start checkout 2008.12.22 06:30:10] >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build >> cvs return value: 0 >> [end checkout 2008.12.22 06:49:53] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 06:49:55 -- compile in progress. >> [start compile 2008.12.22 06:49:55] >> compile return value: 0 >> [end compile 2008.12.22 06:55:09] >> COMPILE_RETURN = 1 >> *** COMPILE FAILED >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" >> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:57:16 -- checkout in progress. >> [start checkout 2008.12.22 06:57:18] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> cvs return value: 0 >> [end checkout 2008.12.22 07:16:39] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 07:16:41 -- compile in progress. >> [start compile 2008.12.22 07:16:41] >> compile return value: 0 >> [end compile 2008.12.22 08:02:05] >> COMPILE_RETURN = 0 >> 2008.12.22 08:02:22 -- tests in progress. >> [start run-tests 2008.12.22 08:02:22] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> [end run-tests 2008.12.22 08:02:22] >> TESTS_RETURN = 0 >> 2008.12.22 08:02:22 -- checkout, compile and test run done. >> >> --- >> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> done. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 23 14:07:20 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 13:07:20 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Message-ID: ah. I thought for some reason that m3cc could use newer stuff, probably confusing with m3tests, and I wouldn't catch this as I back up to older versions rarely. That's /trivial/ to fix..though I'm still hampered by unusual real-world circumstance, should get relief by end of week (Friday), or maybe much earlier, but certainly during Friday day. The fix is to just provide one's own, like: if not defined("fs_exists") proc fs_exists(a) is return state(a, a) end end Thanks, - Jay> Date: Tue, 23 Dec 2008 13:59:00 +0100> From: wagner at elegosoft.com> To: m3devel at elegosoft.com> Subject: Re: [M3devel] Fwd: Output from "cron" command> > Quoting Tony Hosking :> > > Jay, things are still failing on my SOLgnu installation after your> > changes. Did you ever try it on yours?> > According to tinderbox, all release builds fail due to undefined> quake procedures:> > > /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh > -c "cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? " m3cc> 1196 === package m3-sys/m3cc ===> 1197 +++ cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? +++> 1198 --- building in SOLgnu ---> 1199> 1200 NEXT > "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: > fs_exists> 1201> > fs_exists cannot be used unless we provide an official release before,> because it has not been officially released yet.> > Some lastok-builds fail for other reasons.> > Has anybody time to fix these? Alternatively, we could back out the> offending changes...> > > Happy holidays to all!> > Happy Christmas to all M3 users from me, too!> > Olaf> > > Begin forwarded message:> >> >> From: Tony Hosking > >> Date: 23 December 2008 00:04:20 GMT+11:00> >> To: hosking at cs.purdue.edu> >> Subject: Output from "cron" command> >>> >> Your "cron" job on niagara.cs.purdue.edu> >> $HOME/cm3/scripts/regression/cron.sh> >>> >> produced the following output:> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara release-build"> >>> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:30:06 -- checkout in progress.> >> [start checkout 2008.12.22 06:30:10]> >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build> >> cvs return value: 0> >> [end checkout 2008.12.22 06:49:53]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 06:49:55 -- compile in progress.> >> [start compile 2008.12.22 06:49:55]> >> compile return value: 0> >> [end compile 2008.12.22 06:55:09]> >> COMPILE_RETURN = 1> >> *** COMPILE FAILED> >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"> >>> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:57:16 -- checkout in progress.> >> [start checkout 2008.12.22 06:57:18]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> cvs return value: 0> >> [end checkout 2008.12.22 07:16:39]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 07:16:41 -- compile in progress.> >> [start compile 2008.12.22 07:16:41]> >> compile return value: 0> >> [end compile 2008.12.22 08:02:05]> >> COMPILE_RETURN = 0> >> 2008.12.22 08:02:22 -- tests in progress.> >> [start run-tests 2008.12.22 08:02:22]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> [end run-tests 2008.12.22 08:02:22]> >> TESTS_RETURN = 0> >> 2008.12.22 08:02:22 -- checkout, compile and test run done.> >>> >> ---> >>> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> done.> > > > -- > Olaf Wagner -- elego Software Solutions GmbH> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 18:28:36 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 11:28:36 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <49511FC4.2050903@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 2^16-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. I think we already have a reasonably designed abstraction for TEXT, a bit of it built in to the language and the rest in Text.i3. Only problem 1) affects the abstraction. Addressing only 1) for now: It ought to cause minimal grief to just change WIDECHAR so it has a big enough value range for all the Unicode values, probably 32 bits in today's world. Surely nobody has written any code that assumes BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly hard change to adapt old code to, since the static rules of the language would point directly to most places that would need to be fixed. That leaves the wide TEXT literals. It is easy to forget, and I have to keep reminding myself, but (assuming again that nobody has gotten their fingers improperly into the implementation pie) there is currently no such thing as a "WIDETEXT" type. Both kinds of literals are of type TEXT. They are just different lexical rules for specifying literal values of type TEXT. A bit like '16' and '16_10' are different ways of writing the same value, with the same type INTEGER. This differs from the CHAR and WIDECHAR literals, which really are of different types. The one change needed to the W"..." literals would be to allow escape sequences inside for giving characters numerically. Right now, the \x0123 form of escape requires exactly 4 hex digits. If we added a new alternative escape letter, in addition to the 'x', that required, say, exactly 8 hex digits, then these literals could express characters in the needed extra space, without affecting existing code. I suppose, for consistency and completeness, we should also add a new octal escape sequence that was long enough for the full new range. And, we would also need to allow the same new escape sequences in WIDECHAR literals. This would solve the lack of sufficient range problem and have either minimal or zero impact on existing code. Compiler and library changes would not be trivial, but quite reasonable. I believe problems 2) and 3) can be addressed solely by working on the TEXT implementation - Rodney Bates From dragisha at m3w.org Tue Dec 23 19:43:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 19:43:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <49511FC4.2050903@wichita.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> Message-ID: <1230057780.10385.23.camel@faramir.m3w.org> As my mother's tongue uses two alphabets for writing, Latin covered by ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph correspondence and three digraphs in Latin variant I think I have over-average experience with non-Latin1 alphabets, in various areas. If I have to express what we call widetext literal in my code, I will have to work with Unicode tables and pick character by character. Tedious! What I would do is - switch my keyboard to either Latin or Cyrillic mapping and - imagine that!!! - just type! Thus getting UTF-8 characters into my source. My example literal would be: CONST MyNameInCyrillic = "??????? ?????"; MyNameInLatin = "Dragi?a Duri?"; You can see or not these glyphs, depending on your MUA and to some degree on MTA's in transit. With all WIDE* talk it is what I am using. Me being example guy from non-Latin1 world. How many of you are non-Latin1 people and using 16bit "W literals" ? dd On Tue, 2008-12-23 at 11:28 -0600, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values that > exceed 2^16-1. > ... > I think we already have a reasonably designed abstraction for TEXT, > a bit of it built in to the language and the rest in Text.i3. Only > problem 1) affects the abstraction. Addressing only 1) for now: > > It ought to cause minimal grief to just change WIDECHAR so it has a > big enough value range for all the Unicode values, probably 32 bits > in today's world. Surely nobody has written any code that assumes > BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > hard change to adapt old code to, since the static rules of the > language would point directly to most places that would need to be > fixed. > > That leaves the wide TEXT literals. It is easy to forget, and I have > to keep reminding myself, but (assuming again that nobody has gotten > their fingers improperly into the implementation pie) there is > currently > no such thing as a "WIDETEXT" type. Both kinds of literals are of > type > TEXT. They are just different lexical rules for specifying literal > values of type TEXT. A bit like '16' and '16_10' are different ways > of writing the same value, with the same type INTEGER. This differs > from the CHAR and WIDECHAR literals, which really are of different > types. > > The one change needed to the W"..." literals would be to allow > escape sequences inside for giving characters numerically. Right > now, the \x0123 form of escape requires exactly 4 hex digits. If > we added a new alternative escape letter, in addition to the 'x', that > required, say, exactly 8 hex digits, then these literals could express > characters in the needed extra space, without affecting existing code. > I suppose, for consistency and completeness, we should also add a new > octal escape sequence that was long enough for the full new range. > > And, we would also need to allow the same new escape sequences in > WIDECHAR > literals. > ... > > - Rodney Bates > -- Dragi?a Duri? From hendrik at topoi.pooq.com Wed Dec 24 14:00:15 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:00:15 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <20081224130015.GA1001@topoi.pooq.com> On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. I see it. All we need to do to make this work is say that Modula-3 programs are encoded in Unicode. That and some work under the hood so make it work. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? Well, I'm from the Latin-1 world. I've had to debug a compiler that was mistreating Korean strings (this was before Unicode), and I've written a program that used UTF-8 hiragana. But I only visit the nonLatin-1 world. I don't live there. -- hendrik From hendrik at topoi.pooq.com Wed Dec 24 14:10:30 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:10:30 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224130015.GA1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> Message-ID: <20081224131030.GD1001@topoi.pooq.com> On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > CONST > > MyNameInCyrillic = "??????? ?????"; > > MyNameInLatin = "Dragi?a Duri?"; > > > > You can see or not these glyphs, depending on your MUA and to some > > degree on MTA's in transit. > > I see it. All we need to do to make this work is say that Modula-3 > programs are encoded in Unicode. That and some work under the hood so > make it work. Well, probably a *lot* of work. -- hendrik From dragisha at m3w.org Tue Dec 23 20:34:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 20:34:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224131030.GD1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> Message-ID: <1230060840.10385.30.camel@faramir.m3w.org> On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > CONST > > > MyNameInCyrillic = "??????? ?????"; > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > degree on MTA's in transit. > > > > I see it. All we need to do to make this work is say that Modula-3 > > programs are encoded in Unicode. That and some work under the hood so > > make it work. > > Well, probably a *lot* of work. Not at all. I am already using it. Just typing into. What does not work and I am not using it "length in glyphs". Not important for me, but not hard to implement. Either directly in Modula-3 or using external C lib. Search for UTF-8 pattern in UTF-8 text works - of course. Also - partitioning of TEXT's by *any real* criteria. By position of CHAR, or by position of any substring. > > -- hendrik > -- Dragi?a Duri? From mika at async.caltech.edu Tue Dec 23 22:15:39 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 13:15:39 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Tue, 23 Dec 2008 11:28:36 CST." <49511FC4.2050903@wichita.edu> Message-ID: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> "Rodney M. Bates" writes: ... >It ought to cause minimal grief to just change WIDECHAR so it has a >big enough value range for all the Unicode values, probably 32 bits >in today's world. Surely nobody has written any code that assumes >BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly ... Minor issue, I know, but... Pickles? Mika From rodney.bates at wichita.edu Tue Dec 23 23:31:54 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:31:54 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <495166DA.2090606@wichita.edu> Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? Yes, I have been thinking about this too, and some implementation work would be required here. It should not be too hard to use the existing mechanism that pickles/unpickles INTEGER fields of differing sizes on different architectures and do the same thing with different sizes of WIDECHAR. And apparently having had my fingers inside of pickles recently more than anybody else, I would expect to be the one to do it. It is likely that network objects containing such fields would want similar treatment. > > Mika From rodney.bates at wichita.edu Tue Dec 23 23:45:47 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:45:47 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <49516A1B.8050903@wichita.edu> They look very believable on my email client. Perhaps I glossed over this. Yes, if your keyboard/editor can be set to handle the glyphs you want directly, then you can always use them in literals, of either kind, instead of escape sequences. I am sure we need more implementation work here, but I think the TEXT abstraction is fine in this respect. We still need escapes too, as an alternative for when using a keyboard/display setting isn't convenient, and it is for this purpose I am suggesting this extension to the TEXT literals, as part of making Unicode support complete. Dragi?a Duri? wrote: > As my mother's tongue uses two alphabets for writing, Latin covered by > ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph > correspondence and three digraphs in Latin variant I think I have > over-average experience with non-Latin1 alphabets, in various areas. > > If I have to express what we call widetext literal in my code, I will > have to work with Unicode tables and pick character by character. > Tedious! > > What I would do is - switch my keyboard to either Latin or Cyrillic > mapping and - imagine that!!! - just type! Thus getting UTF-8 characters > into my source. My example literal would be: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? > - Rodney Bates From rodney.bates at wichita.edu Wed Dec 24 00:22:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 17:22:35 -0600 Subject: [M3devel] This disgusting TEXT business Message-ID: <495172BB.9070906@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 216-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. As for 2), the implementation could be greatly improved without changing the existing TEXT abstraction. It could even be improved a lot without changing the existing data structures, only the algorithms. Back in March, I conjectured about improving get_chars for a concatenation by recursing on only one side and iterating on the other, but I never did anything about it. Text.Cat could flatten concatenations that are short into one of the atomic representations. It could perhaps do a two-level rebalance of concatenation trees. All these would help at least somewhat with the performance consequences of building a string one character at a time by concatenation. And of course, ignoring the needs of those who have violated the TEXT abstraction, we could come up with entirely new internal representations. We could even just add some new ones and support mixtures of both the old and new ones. This would really just be an expansion of what we already have, which has several representations, with all the operations dynamically checking and adapting to them. On the other hand, I do know from hard experience, that the implementation size and complexity go up surprisingly as the number of alternative representations goes up. It's really a sort of cartesion product of operations, operands for each operation, and representations for each operand. Something that has not been mentioned is the _space_ overhead of the existing concatenation scheme. Proponents of the pure or nearly-pure OO language design philosophy (e.g. Smalltalk, Java) seem pretty oblivious to how much you lose with lots of separately-allocated heap objects connected by pointers. Aside from the extra pointers, there is allocator/GC overhead per object and fragmentation loss. Then there's loss of reference locality, leading to bigger working sets, which spills back into time overhead. And with heap-allocated open arrays, there are additional 1+NoOfDimensions words as well. So I am strongly in favor of cutting down on the number of separate objects, wherever reasonable. Rodney Bates From hendrik at topoi.pooq.com Wed Dec 24 19:00:01 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 13:00:01 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230060840.10385.30.camel@faramir.m3w.org> References: <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> <1230060840.10385.30.camel@faramir.m3w.org> Message-ID: <20081224180001.GA1449@topoi.pooq.com> On Tue, Dec 23, 2008 at 08:34:00PM +0100, Dragi?a Duri? wrote: > On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > > > CONST > > > > MyNameInCyrillic = "??????? ?????"; > > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > > degree on MTA's in transit. > > > > > > I see it. All we need to do to make this work is say that Modula-3 > > > programs are encoded in Unicode. That and some work under the hood so > > > make it work. > > > > Well, probably a *lot* of work. > > Not at all. I am already using it. Just typing into. > > What does not work and I am not using it "length in glyphs". Not > important for me, but not hard to implement. Either directly in Modula-3 > or using external C lib. > > Search for UTF-8 pattern in UTF-8 text works - of course. Also - > partitioning of TEXT's by *any real* criteria. By position of CHAR, or > by position of any substring. I see. You are just using the existing TEXT to store the bytes of UTF-8. That seems one of the simplest ways of accomodating UTF-8, in fact. No conversions in or out, just work in a pure UTF-8 environment. The "work" I talked about would be the effert necessary to implement all the existing methods that use character counts as character counts. Treating them as using byte counts is indeed simple. What remains is to build operations that deal in these UTF-8 TEXTs as being strings of UTF-8 characters, with operations like fetching and storing characters instead of bytes. Perhaps what is really needed is clarity in the specification, so we know which operations to use on bytes, and which on characters, and abolish the myth that character counts are an efficient way of accessing characters. In retrospect, maybe there should have been a type BYTE, a type CHARACTER, and a type TEXT, all conceptually separate. TEXT could be accessed by byte or character operations; and indexes into TEXT would be byte offsets. This lacks some generality that might have been needed in the days of machines with word-addressable 29-bit words, but those days seem to be past. What seems to block introducing this approach seems to be the prevalence of code that uses CHARACTER as a synonym for BYTE. Do we, for some kind of compatibility, have to use CHARACTER when we mean BYTE, and something else when we want to say CHARACTER? In the long run, will be be more inconvenienced by having to rewrite code that uses CHARACTER to represent characters, or code that uses CHARACTER to represent bytes? It is already possible to use 0..255 to represent bytes, isn't it? Or am I wrong in this? Might this mean four-byte integers that happen to have a restricted range of values? -- hendrik From darko at darko.org Wed Dec 24 01:20:27 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:20:27 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <495172BB.9070906@wichita.edu> References: <495172BB.9070906@wichita.edu> Message-ID: <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> I agree with your list of problems, but you don't need a cartesian product of operations for string operations between multiple representations. All you need is iteration through the characters of a string and concatenation with another string for each representation. The object interface I proposed is little more than a wrapper for the existing Text interface plus the iteration method. The getData and setData methods in that object provide an interface to external APIs. On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values > that > exceed 216-1. > > 2) The CM3 TEXT implementation has serious inefficiencies in at > least some > realistic cases. > > 3) We want some kind of compatibility with other software. > > As for 2), the implementation could be greatly improved without > changing > the existing TEXT abstraction. It could even be improved a lot > without > changing the existing data structures, only the algorithms. > > Back in March, I conjectured about improving get_chars for a > concatenation > by recursing on only one side and iterating on the other, but I > never did > anything about it. Text.Cat could flatten concatenations that are > short > into one of the atomic representations. It could perhaps do a two- > level > rebalance of concatenation trees. All these would help at least > somewhat > with the performance consequences of building a string one character > at > a time by concatenation. > > And of course, ignoring the needs of those who have violated the TEXT > abstraction, we could come up with entirely new internal > representations. > We could even just add some new ones and support mixtures of both > the old > and new ones. This would really just be an expansion of what we > already > have, which has several representations, with all the operations > dynamically > checking and adapting to them. > > On the other hand, I do know from hard experience, that the > implementation > size and complexity go up surprisingly as the number of alternative > representations goes up. It's really a sort of cartesion product of > operations, operands for each operation, and representations for each > operand. > > Something that has not been mentioned is the _space_ overhead of the > existing concatenation scheme. Proponents of the pure or nearly-pure > OO language design philosophy (e.g. Smalltalk, Java) seem pretty > oblivious to how much you lose with lots of separately-allocated > heap objects connected by pointers. > > Aside from the extra pointers, there is allocator/GC overhead > per object and fragmentation loss. Then there's loss of > reference locality, leading to bigger working sets, which spills > back into time overhead. And with heap-allocated open arrays, > there are additional 1+NoOfDimensions words as well. So I am > strongly in favor of cutting down on the number of separate objects, > wherever reasonable. > > Rodney Bates From darko at darko.org Wed Dec 24 01:27:20 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:27:20 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Besides breaking pickles (which isn't minor to those it affects) it also has the effect of doubling the size of String16 arrays and any wide strings or arrays. Why change the meaning of an existing type, possibly breaking unknown software when we can just define another type? It's not like another built-in type will cause anyone problems. On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? > > Mika From mika at async.caltech.edu Wed Dec 24 01:29:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:29:15 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:27:20 +0900." <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Message-ID: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Darko writes: >Besides breaking pickles (which isn't minor to those it affects) it >also has the effect of doubling the size of String16 arrays and any >wide strings or arrays. Why change the meaning of an existing type, >possibly breaking unknown software when we can just define another >type? It's not like another built-in type will cause anyone problems. stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > >> "Rodney M. Bates" writes: >> ... >>> It ought to cause minimal grief to just change WIDECHAR so it has a >>> big enough value range for all the Unicode values, probably 32 bits >>> in today's world. Surely nobody has written any code that assumes >>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >> ... >> >> Minor issue, I know, but... Pickles? >> >> Mika From mika at async.caltech.edu Wed Dec 24 01:31:50 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:31:50 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:20:27 +0900." <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> Message-ID: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Might I suggest an interface somewhat like the Rd/Wr interfaces? One could provide a buffer into which characters are read en masse instead of going through the method calls for each character.. Of course for equality this isn't necessarily more efficient since you often can do an early-out: although I believe that Text.Equal (when performance matters) usually does return TRUE.... I think it is important that basic TEXT operations be very, very fast. Many people care about that. Many of those that don't wouldn't be using a compiled language anyhow. Mika Darko writes: >I agree with your list of problems, but you don't need a cartesian >product of operations for string operations between multiple >representations. All you need is iteration through the characters of a >string and concatenation with another string for each representation. >The object interface I proposed is little more than a wrapper for the >existing Text interface plus the iteration method. The getData and >setData methods in that object provide an interface to external APIs. > > >On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > >> I hear three problems with CM3 TEXT: >> >> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >> that >> exceed 216-1. >> >> 2) The CM3 TEXT implementation has serious inefficiencies in at >> least some >> realistic cases. >> >> 3) We want some kind of compatibility with other software. >> >> As for 2), the implementation could be greatly improved without >> changing >> the existing TEXT abstraction. It could even be improved a lot >> without >> changing the existing data structures, only the algorithms. >> >> Back in March, I conjectured about improving get_chars for a >> concatenation >> by recursing on only one side and iterating on the other, but I >> never did >> anything about it. Text.Cat could flatten concatenations that are >> short >> into one of the atomic representations. It could perhaps do a two- >> level >> rebalance of concatenation trees. All these would help at least >> somewhat >> with the performance consequences of building a string one character >> at >> a time by concatenation. >> >> And of course, ignoring the needs of those who have violated the TEXT >> abstraction, we could come up with entirely new internal >> representations. >> We could even just add some new ones and support mixtures of both >> the old >> and new ones. This would really just be an expansion of what we >> already >> have, which has several representations, with all the operations >> dynamically >> checking and adapting to them. >> >> On the other hand, I do know from hard experience, that the >> implementation >> size and complexity go up surprisingly as the number of alternative >> representations goes up. It's really a sort of cartesion product of >> operations, operands for each operation, and representations for each >> operand. >> >> Something that has not been mentioned is the _space_ overhead of the >> existing concatenation scheme. Proponents of the pure or nearly-pure >> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >> oblivious to how much you lose with lots of separately-allocated >> heap objects connected by pointers. >> >> Aside from the extra pointers, there is allocator/GC overhead >> per object and fragmentation loss. Then there's loss of >> reference locality, leading to bigger working sets, which spills >> back into time overhead. And with heap-allocated open arrays, >> there are additional 1+NoOfDimensions words as well. So I am >> strongly in favor of cutting down on the number of separate objects, >> wherever reasonable. >> >> Rodney Bates From darko at darko.org Wed Dec 24 01:35:37 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:35:37 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> References: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Message-ID: <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Those don't involve data stored in files, can be fixed with a recompile and are only an issue if you reveal the object implementation of TEXT. On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > Darko writes: >> Besides breaking pickles (which isn't minor to those it affects) it >> also has the effect of doubling the size of String16 arrays and any >> wide strings or arrays. Why change the meaning of an existing type, >> possibly breaking unknown software when we can just define another >> type? It's not like another built-in type will cause anyone problems. > > stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >> >> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >> >>> "Rodney M. Bates" writes: >>> ... >>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>> big enough value range for all the Unicode values, probably 32 bits >>>> in today's world. Surely nobody has written any code that assumes >>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>> ... >>> >>> Minor issue, I know, but... Pickles? >>> >>> Mika From mika at async.caltech.edu Wed Dec 24 01:41:20 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:41:20 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:35:37 +0900." <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Message-ID: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Sorry, I think I misinterpreted what you meant by "built-in type". Darko writes: >Those don't involve data stored in files, can be fixed with a >recompile and are only an issue if you reveal the object >implementation of TEXT. > >On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > >> Darko writes: >>> Besides breaking pickles (which isn't minor to those it affects) it >>> also has the effect of doubling the size of String16 arrays and any >>> wide strings or arrays. Why change the meaning of an existing type, >>> possibly breaking unknown software when we can just define another >>> type? It's not like another built-in type will cause anyone problems. >> >> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >> >>> >>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>> >>>> "Rodney M. Bates" writes: >>>> ... >>>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>>> big enough value range for all the Unicode values, probably 32 bits >>>>> in today's world. Surely nobody has written any code that assumes >>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>>> ... >>>> >>>> Minor issue, I know, but... Pickles? >>>> >>>> Mika From darko at darko.org Wed Dec 24 01:47:36 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:47:36 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> References: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Message-ID: <064090A2-A17E-4CBF-9B7B-3A5B575C5706@darko.org> The proposed interface is like Rd/Wr in that it has a procedural interface on an opaque type that is revealed to be implemented as an object for each type of Rd/Wr. But the idea is to avoid the "one true" representation (which what the buffer is) because efficiency means different things to different people. If you do a lot of Unicode text and an UTF-32 representation is probably fastest. If you want to save space or you deal mostly with ASCII then UTF-8 is probably best. Going through each character is needed if you don't know the representation of the string you are dealing with. If you do know the format of the string you can do other operations. Since most applications will use one representation you will mostly be doing operations between strings of the same representations, which will be very fast, for instance you might call a local OS API to do the work. Having application defined representations means that you can adapt the string representation and operations to what you need to give you the performance you're after, or to choose one from an existing library that works best. On 24/12/2008, at 9:31 AM, Mika Nystrom wrote: > Might I suggest an interface somewhat like the Rd/Wr interfaces? > One could provide a buffer into which characters are read en masse > instead of going through the method calls for each character.. > > Of course for equality this isn't necessarily more efficient since > you often can do an early-out: although I believe that Text.Equal > (when performance matters) usually does return TRUE.... > > I think it is important that basic TEXT operations be very, very > fast. Many people care about that. Many of those that don't > wouldn't be using a compiled language anyhow. > > Mika > > Darko writes: >> I agree with your list of problems, but you don't need a cartesian >> product of operations for string operations between multiple >> representations. All you need is iteration through the characters >> of a >> string and concatenation with another string for each representation. >> The object interface I proposed is little more than a wrapper for the >> existing Text interface plus the iteration method. The getData and >> setData methods in that object provide an interface to external APIs. >> >> >> On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: >> >>> I hear three problems with CM3 TEXT: >>> >>> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >>> that >>> exceed 216-1. >>> >>> 2) The CM3 TEXT implementation has serious inefficiencies in at >>> least some >>> realistic cases. >>> >>> 3) We want some kind of compatibility with other software. >>> >>> As for 2), the implementation could be greatly improved without >>> changing >>> the existing TEXT abstraction. It could even be improved a lot >>> without >>> changing the existing data structures, only the algorithms. >>> >>> Back in March, I conjectured about improving get_chars for a >>> concatenation >>> by recursing on only one side and iterating on the other, but I >>> never did >>> anything about it. Text.Cat could flatten concatenations that are >>> short >>> into one of the atomic representations. It could perhaps do a two- >>> level >>> rebalance of concatenation trees. All these would help at least >>> somewhat >>> with the performance consequences of building a string one character >>> at >>> a time by concatenation. >>> >>> And of course, ignoring the needs of those who have violated the >>> TEXT >>> abstraction, we could come up with entirely new internal >>> representations. >>> We could even just add some new ones and support mixtures of both >>> the old >>> and new ones. This would really just be an expansion of what we >>> already >>> have, which has several representations, with all the operations >>> dynamically >>> checking and adapting to them. >>> >>> On the other hand, I do know from hard experience, that the >>> implementation >>> size and complexity go up surprisingly as the number of alternative >>> representations goes up. It's really a sort of cartesion product of >>> operations, operands for each operation, and representations for >>> each >>> operand. >>> >>> Something that has not been mentioned is the _space_ overhead of the >>> existing concatenation scheme. Proponents of the pure or nearly- >>> pure >>> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >>> oblivious to how much you lose with lots of separately-allocated >>> heap objects connected by pointers. >>> >>> Aside from the extra pointers, there is allocator/GC overhead >>> per object and fragmentation loss. Then there's loss of >>> reference locality, leading to bigger working sets, which spills >>> back into time overhead. And with heap-allocated open arrays, >>> there are additional 1+NoOfDimensions words as well. So I am >>> strongly in favor of cutting down on the number of separate objects, >>> wherever reasonable. >>> >>> Rodney Bates From darko at darko.org Wed Dec 24 01:50:53 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:50:53 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> References: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Message-ID: Sorry I thought you were referring to TEXT representations. We could define UNICHAR = INTEGER, an alias, which would avoid those changes. That might be required since the Unicode character set seems to change constantly and it might not work to define it as an enumeration. On 24/12/2008, at 9:41 AM, Mika Nystrom wrote: > > Sorry, I think I misinterpreted what you meant by "built-in type". > > Darko writes: >> Those don't involve data stored in files, can be fixed with a >> recompile and are only an issue if you reveal the object >> implementation of TEXT. >> >> On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: >> >>> Darko writes: >>>> Besides breaking pickles (which isn't minor to those it affects) it >>>> also has the effect of doubling the size of String16 arrays and any >>>> wide strings or arrays. Why change the meaning of an existing type, >>>> possibly breaking unknown software when we can just define another >>>> type? It's not like another built-in type will cause anyone >>>> problems. >>> >>> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >>> >>>> >>>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>>> >>>>> "Rodney M. Bates" writes: >>>>> ... >>>>>> It ought to cause minimal grief to just change WIDECHAR so it >>>>>> has a >>>>>> big enough value range for all the Unicode values, probably 32 >>>>>> bits >>>>>> in today's world. Surely nobody has written any code that >>>>>> assumes >>>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a >>>>>> terribly >>>>> ... >>>>> >>>>> Minor issue, I know, but... Pickles? >>>>> >>>>> Mika From jay.krell at cornell.edu Wed Dec 24 09:00:39 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 24 Dec 2008 08:00:39 +0000 Subject: [M3devel] a text proposal?? Message-ID: NOTE I am not a vehement advocate of the below. I'm at least "floating" a "thought balloon". Like, doesn't this sound fairly ok and reasonable? I propose the following "one" "parameterized" representation: a block of memory with a stored allocated size, probably in bytes, but could be divided by character size (i.e. "length in chars" vs. "length in bytes") a stored character size, in bytes, which shall be 1, 2, or 4 a stored length, probably in bytes, but could be divided by character size a stored maximum character value, maybe This is too expensive to maintain in general. It could be "advisory". However, it is also somewhat redundant with the character size. There shall be one implementation, though part of it shall be in a generic module, parameterized by character size. (yeah, yeah, parameterized by whatever Modula-3 allows) Appending a string with character size n with a string with character size m shall result in a string with character size max(n, m). String append shall always result in a "flat array". Never linked lists. Strings are never "encoded". Characters shall only ever be zero extended. Maybe there shall be functions to create strings from encoded strings. e.g. from utf8 and whatever is the name for 16 bit unicode that contain 20 bit surrogate pairs or whatever they are called. And to encode strings. Concatenating shall grow the allocated size by at least a constant factor greater than one. Possibly 2, possibly smaller. If that fails, fallback to allocating only what is needed. [This probably makes no sense, given text immutability and that concat produces new texts. But it might make sense if "text" is a value type.] Concatenation of n texts shall be a primitive, not just 2 texts. [It looks like this is already the case.] I think TEXTs are considered immutable. Over-allocating for later in-place concatenation doesn't, I believe, contract that. However perhaps I misspoke, and the allocated size is stored with the pointer, and the text contains a pointer to that, and a length. And maybe an offset. But "internal pointers" are ok and therefore an offset not needed, right? Cop out rationale: Anyone who needs O(1) concatenation of very long strings, is probably using the wrong type. Since texts are immutable, it may be viable, or even cheap, to allow a text to stick around with multiple representations, possibly using "weak refs". That is, string shall have probably a function or functions to get at the raw chars. Either the user has to specify a char size he wants, and get back null if the wrong size, or perhaps if user asks for larger than is needed, the text realizes that representation on-demand? And then it sticks around, in case another client soon thereafter asks for the same wider-than-necessary representation? The first order implementation can be simpler though. It can either return null for all but the current representation, or realize and return, but not hold onto, whatever representation is requested. Note that I'm not familiar with the code (yet) so this might be partly or total nonsense. I'm particular uncertain about the ability to double allocation size as individual chars are appended to a text. Issues of normalization remain punted. They already exist. Functions could be provided for convert to composed, decomposed, etc. - Jay From dragisha at m3w.org Thu Dec 25 01:00:05 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 25 Dec 2008 01:00:05 +0100 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 Message-ID: <1230163205.7131.1.camel@faramir.m3w.org> Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode (via UTF-8 as internal rep) then we must also extend CHAR so it can represent any single Unicode glyph ? in fact CHAR becomes 32bit value instead of current 8bit. If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we are on UNICHAR route ? as proposed by Darko (IIRC). But ? down that road we have variations in very traditional interface Text.i3 ? Text.GetChar, Text.SetChars, Text.FromChar, Text.FromChars must have these two variants ? somehow. I see no elegant way to handle that if we insist on BITSIZE(CHAR) = 8. UNICHAR route also contains other branches, most of them analog to current Text8/Text16 mess. UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte buffers, is one problem. Not too hard to spot and fix, though. Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that way. If similar need happens in future, ie some runtime level data communication, I think we can do it at ?connection? level. Some marshalling would most probably always take place ? so why not add TEXT I/O to list of tasks needed? -- Dragi?a Duri? From dabenavidesd at yahoo.es Thu Dec 25 01:12:07 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 25 Dec 2008 00:12:07 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Dear all, actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the? helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix INTERFACE). In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. You can read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess)? http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on? back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32..m3.html ). Happy holidays and hope this helps a bit more. --- El lun, 22/12/08, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Para: "Jay" CC: mika at camembert.async.caltech.edu Fecha: lunes, 22 diciembre, 2008 10:49 I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now
-- changing my "=
>ino_t" to
"m3_ino_t".
I'm still
checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 -
 Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinbishop at bellsouth.net Thu Dec 25 03:14:31 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Wed, 24 Dec 2008 20:14:31 -0600 Subject: [M3devel] Modula-3 subreddit Message-ID: <4952EC87.5050709@bellsouth.net> Just wanted to say that I made a Modula-3 subreddit on Reddit (a while back) if anyone is interested in joining (and hopefully submitting interesting links :P) http://www.reddit.com/r/modula3/ From mika at async.caltech.edu Fri Dec 26 09:56:24 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 26 Dec 2008 00:56:24 -0800 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 In-Reply-To: Your message of "Thu, 25 Dec 2008 01:00:05 +0100." <1230163205.7131.1.camel@faramir.m3w.org> Message-ID: <200812260856.mBQ8uOZ0065080@camembert.async.caltech.edu> I think you may find "SET OF CHAR" hidden all over the place. Also a lot of code may depend on the fact that CHAR and C's "char" types match on a given platform. This is your UNSAFE objection, but I think it goes further than just buffers. Mika =?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode >(via UTF-8 as internal rep) then we must also extend CHAR so it can >represent any single Unicode glyph ??? in fact CHAR becomes 32bit value >instead of current 8bit. > >If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we >are on UNICHAR route ??? as proposed by Darko (IIRC). But ??? down that road >we have variations in very traditional interface Text.i3 ??? Text.GetChar, >Text.SetChars, Text.FromChar, Text.FromChars must have these two >variants ??? somehow. I see no elegant way to handle that if we insist on >BITSIZE(CHAR) = 8. > >UNICHAR route also contains other branches, most of them analog to >current Text8/Text16 mess. > >UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte >buffers, is one problem. Not too hard to spot and fix, though. > >Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that >way. If similar need happens in future, ie some runtime level data >communication, I think we can do it at ???connection??? level. Some >marshalling would most probably always take place ??? so why not add TEXT >I/O to list of tasks needed? > >-- >Dragi??a Duri?? From martinbishop at bellsouth.net Sat Dec 27 03:39:08 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Fri, 26 Dec 2008 20:39:08 -0600 Subject: [M3devel] Who runs modula3.org? Message-ID: <4955954C.5050102@bellsouth.net> Who runs modula3.org, or at least has access? Quite a few of the links are dead. Some link to m3.org, others link to gatekeeper (which is gone). Most have alternative pages hosted elsewhere, but almost all are archived at archive.org, if needed. From dragisha at m3w.org Sat Dec 27 08:02:11 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 27 Dec 2008 08:02:11 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <4955954C.5050102@bellsouth.net> References: <4955954C.5050102@bellsouth.net> Message-ID: <1230361331.7131.6.camel@faramir.m3w.org> Public reply, so I can be corrected by someone knowing better. WWW is AFAIK run by Elego, but content is version controlled from CVS, so IMO it can be changed usual way - by editing and commiting. I don't know about permissions and update procedures for this, but you can browse src repo a bit and see for yourself. On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: > Who runs modula3.org, or at least has access? Quite a few of the links > are dead. Some link to m3.org, others link to gatekeeper (which is > gone). Most have alternative pages hosted elsewhere, but almost all are > archived at archive.org, if needed. -- Dragi?a Duri? From wagner at elegosoft.com Sat Dec 27 15:43:49 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 27 Dec 2008 15:43:49 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <1230361331.7131.6.camel@faramir.m3w.org> References: <4955954C.5050102@bellsouth.net> <1230361331.7131.6.camel@faramir.m3w.org> Message-ID: <20081227154349.5lyajia0w0k88wko@mail.elegosoft.com> Quoting Dragi?a Duri? : > Public reply, so I can be corrected by someone knowing better. > > WWW is AFAIK run by Elego, but content is version controlled from CVS, > so IMO it can be changed usual way - by editing and commiting. > > I don't know about permissions and update procedures for this, but you > can browse src repo a bit and see for yourself. > > On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: >> Who runs modula3.org, or at least has access? Quite a few of the links >> are dead. Some link to m3.org, others link to gatekeeper (which is >> gone). Most have alternative pages hosted elsewhere, but almost all are >> archived at archive.org, if needed. Elego took over the hosting of modula3.org from Igencorp (formerly Critical Mass) run by Farshad Nayeri one or two years ago, I think. The sources are checked in somewhere on birch.elego.de (though I don't know the exact path offhand (I'm not at home right now)) and can be edited and shipped by anyone with ssh access to birch. You can also send me diffs and I'll try to apply them, which may take some days of course. Any improvements and corrections are very welcome! Olaf > -- > Dragi?a Duri? > > -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Sat Dec 27 20:48:46 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 19:48:46 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 Message-ID: fyi: I've been looking at Utime.i3. One of the variations is whether or not struct_tm contains gmtoff and zone at the end. C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 are wrong on some systems. They unconditionally add the two fields, and read from memory that doesn't necessarily have them. It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). I'll double check Solaris and Irix soon or next week. I'll try to come up with a fix shortly. It will surely involve deleting those two files. And breaking any users of Utime.localtime and Utime.gmtime. - Jay From jay.krell at cornell.edu Sat Dec 27 21:12:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 20:12:50 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 In-Reply-To: References: Message-ID: Are localtime_r and gmtime_r really not widely available these days? I don't have a Darwin system handy this week. That is, we have DateBsd.m3, DateLinux.m3, DatePosix.m3. Excerpt: readonly _DateImpls = { "AMD64_DARWIN": "DateBsd", "AMD64_LINUX": "DateBsd", "FreeBSD*" : "DateBsd", "LINUXLIBC6" : "DateBsd", "MIPS64_OPENBSD" : "DateBsd", "NT386" : "DateLinux", "NT386GNU" : "DateLinux", "SOLgnu" : "DatePosix", "SOLsun" : "DatePosix", "SPARC" : "DateBsd", "*_LINUX" : "DateBsd", "SPARC64_SOLARIS" : "DatePosix", "*_DARWIN" : "DateBsd", "*_OPENBSD" : "DateBsd", } DateLinux lacks reentreent functions, and lacks gmtoff/zone in struct tm. It is only used by Cygwin. And Linux 1.x which I assume doesn't matter. This appears to be an oversight by me, the reentrant functions exist. DateBsd lacks reentrant functions, and has gmtoff/zone in struct tm. It is very widely used. DatePosix has reentrant functions, and lacks gmtoff/zone in struct tm. I rather suspect that everyone can just use DatePosix. ? Or at least that DateBsd can use reentrant functions. Eliminating unnecessary locks/globals always seems good. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sat, 27 Dec 2008 19:48:46 +0000 > Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 > > > fyi: > > > I've been looking at Utime.i3. > One of the variations is whether or not struct_tm contains gmtoff and zone at the end. > > > C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 > C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 > > > are wrong on some systems. > They unconditionally add the two fields, and read from memory that doesn't necessarily have them. > > > It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. > It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). > I'll double check Solaris and Irix soon or next week. > > > I'll try to come up with a fix shortly. > It will surely involve deleting those two files. > And breaking any users of Utime.localtime and Utime.gmtime. > > > - Jay From mika at async.caltech.edu Tue Dec 23 04:49:25 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 22 Dec 2008 19:49:25 -0800 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: Your message of "Sun, 21 Dec 2008 17:57:00 GMT." Message-ID: <200812230349.mBN3nPem008337@camembert.async.caltech.edu> I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now -- changing my "=
>ino_t" to
"m3_ino_t".
I'm still checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 - Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- From jay.krell at cornell.edu Sun Dec 28 01:19:24 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:19:24 +0000 Subject: [M3devel] PPC_LINUX Message-ID: I think PPC_LINUX has been broken for a few years, and not released since 5.2.6. It should be ok now and current 5.2.7 min and std distributions are at http://modula3.elegosoft.com/cm3/uploaded-archives We might want to fix user threads? - Jay From jay.krell at cornell.edu Sun Dec 28 01:22:20 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:22:20 +0000 Subject: [M3devel] port names? Message-ID: Again..what do people want for port names? MSDOS? Nice and short in the old style. I386_MSDOS? I think so. DJGPP? Nice and short in the old style. I386_DJGPP? Good because it identifies more -- the compiler, linker, C runtime. As opposed to say "C_WATCOM_DOS" or "I386_DOS_WATCOM" or such. DOS? Again, nice and short in the old style. I386_SOLARIS? Presumbly. X86_SOLARIS? AMD32_SOLARIS? :) IA32_SOLARIS? - Jay From jay.krell at cornell.edu Sun Dec 28 12:00:11 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 11:00:11 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <892265.61271.qm@web28603.mail.ukl.yahoo.com> References: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Message-ID: another take on this.. Cloning C headers in Modula-3 is both error prone and unsafe. It is not subject to static checking, is not verifiably correct, and errors lead to the same sort of type system violations that unsafe code can cause. I have found a few bugs here and there in the cloned headers, mostly wrt alignment. I replaced it with something probably still unsafe, but far less error prone. - Jay ________________________________ > Date: Thu, 25 Dec 2008 00:12:07 +0000 > From: dabenavidesd at yahoo.es > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > > Dear all, > actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. > I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix > INTERFACE). > In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most > operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). > Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. > Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. > You can > read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . > Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess) http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32.m3.html ). > > Happy holidays and hope this helps a bit more. > > > --- El lun, 22/12/08, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > Para: "Jay" > CC: mika at camembert.async.caltech.edu > Fecha: lunes, 22 diciembre, 2008 > 10:49 > > > I think Daniel has a point. He said "UNSAFE", not > "unsafe". "UNSAFE" > means something very specific in Modula-3, and operations that are > UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and > hence only can be used in UNSAFE MODULEs. > > If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that > is a problem that really shouldn't be allowed to get worse. > > This is all covered in sections 1.4.5 and 2.1 of the Green Book. > I'm not sure exactly what you guys are talking about but I would > appreciate it if I could rely on the Green Book for programming in > Modula-3.... > > Not everything that's is UNSAFE, by the way. It is > perfectly possible for a routine coded in C or Fortran to export a > non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can > definitely export an UNSAFE interface. > > I think if you interpret section 2.1 a bit loosely, > but > I think > still within the spirit of the authors' intent, you can go as far > as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE > interfaces) as long as the program needs to use at least one UNSAFE > INTERFACE in order to trigger the error. The contrapositive of > this is that it should not be possible for a Modula-3 program that > imports only non-UNSAFE interfaces to cause an unchecked runtime > error. Ever. > > Some examples...., in the following, assume that CrashHorribly > can cause an unchecked runtime error when called. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > This is OK since an importer has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT > A; > PROCEDURE CrashHorribly(arg : A.T); END C. > > Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > Might be OK, as long as the only way to get an unchecked runtime > error in C is by passing in a B.T. If you can get an unchecked > runtime error in C.CrashHorribly without importing B (say by passing > in a bare A.T or a "home-made" subtype of A), it's wrong. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C.. > > Definitely wrong. > > Mika > > > Jay writes: >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: > > quoted-printable >> >> >>The real world is unsafe.Safe areas are created by unsafe code=2C which > mus= >>t be=2C to a certain extent=2C bug free=2C in order to provide as safe as > a= >>dvertised. I don't believe what I added is any more untrusted/unsafe > than = >>what was there before.Well=2C maybe in a minor way. m3core has > "lots" of "= >>external" functions=2C which are called by "lots" of > functions at higher le= >>vels..ok=2C maybe not "lots"=2C but if you look at the > m3core/src/unix dire= >>ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C > linux-= >>common -- this is a fairly minimal set of stuff that is being used by > preex= >>isting code. A LITTLE bit I added beyond what was there. If you look at > all= >> the other directories=2C you'll find lots more external stuff=2C most > of w= >>hich is not used within the CM3 codebase (but could > be > used by folks > outsid= >>e). It is true I may have introduced external where it wasn't > previously= >>=2C but I don't think it matters.It is prevalent one level down. Moving > som= >>e of it one level up is ok I believe. =20 >>Non-Posix does potentially have a place in .c files.It > depends.Basically=2C= >> as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes > = >>hard to read=2C and at some point=2C you split the files into separate > file= >>s=2C "no" #ifdefs -- or conceptually each file can be viewed as > wrapped in = >>an #ifdef or other platform detection.You need #ifdef in any C that uses > 64= >> bit integers=2C for example. Current Microsoft compilers do support > "long = >>long" but __int64 supporg goes way back=2C and __int64 is a better > name any= >>way. >>What is the 128 bit integer type to be called -- "__int128" or > "long long > > l= >>ong". >>=20 >>=20 >>The names "char"=2C "short"=2C "int"=2C > "long"=2C "long long" are mostly ju= >>st funny mistaken names. >>The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >>=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > types= >> such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >>=20 >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >>=20 >> - Jay >> >> >> >>Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: > m3dev= >>el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin > was > Re: > [M3commit]= >> FilePosixC.c >> >> >> >>Dear all:I'm glad that Jay and all the people around are making > contributio= >>ns=2C so we can claim that as some day there were a good number of > supporte= >>d platforms nowadays we also have.However I have a question=2C but its > more= >> a suggestion=2C the modification Tony is referring is not as standard as > w= >>e would like since the type safety is affected with the change introduced > a= >>llowing (untrusted) functions in the INTERFACE > FilePosix=2C it= >>'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C > so = >>we may want to mark as UNSAFE INTERFACE=2C however doing that I think some > = >>clients could not claim any more they are safe so that "fix" is > not feasib= >>le.Can you push to a lower level (as in the UNSAFE FilePosix > implementation= >> module > before > committing the change) or make it available in other > Interfa= >>ce already available=2C why do we need that change anyway? Also IMHO we > sh= >>ould leave out what is not POSIX (we should not care for WIN64 although I > k= >>now is needed by the C compiler preprocessor) out of that file.Although I > d= >>on't have the formal specification or program=2C but I can remember > libm3 c= >>ode was specified and mechanically checked with the Extended Static > Checker= >> Modula-3 (which is not sound also=2C that is could be cheated=2C see > http:= >>//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 > )=2C = >> that would be a good reason for keeping as safe as possible and close to > h= >>istorical interfaces and just rework implementations (even if they are not > = >>safe) as needed. I know that implementation is not safe but at least > having= >> it well > enough > implemented the ESC/M3 checker and the compiler would tell > = >>us the library should work Ok and is safe and (hopefully) without the > error= >>s detected by the ESC as we care for the details in the UNSAFE > implementati= >>on.There are any special interfaces we should have as "standard" > in the low= >>er level runtime (hard enough question I know)=2C can we give some > hints/id= >>eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay >>>ll.edu> escribi=F3: >>De: Jay Asunto: Re: [M3devel] > serial/CygwinPara: "m3= >>devel" =2C "Tony" > Fecha: vier= >>nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ > it is o= >>k now -- changing my "ino_t" to"m3_ino_t".I'm still > checking though.And I o= >>nly checked SOLgnu. I will check SOLsun. - > Jay > = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>The real world is unsafe. Safe areas are created by unsafe code=2C > which= >> must be=2C to a certain extent=2C bug free=2C in order to provide as safe > = >>as advertised. =3B =3B I > don't believe what I added is a= >>ny more untrusted/unsafe than what was there before. Well=2C maybe > in a = >>minor way. =3B =3B m3core > has > "lots" of > "external" funct= >>ions=2C which are called by "lots" of functions at higher > levels..ok=2C may= >>be not "lots"=2C but if you look at the m3core/src/unix > directories that I = >>have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this > = >>is a fairly minimal set of stuff that is being used by preexisting code. A > = >>LITTLE bit I added beyond what was there. If you look at all the other > dire= >>ctories=2C you'll find lots more external stuff=2C most of which is not > use= >>d within the CM3 codebase (but could be used by folks > outside). =3B= >> =3B It is true I may have introduced external > where it wasn't p= >>reviously=2C but I don't think it matters. It is prevalent one > level dow= >>n. Moving some of it one level up is ok > I > believe. =3B =3B >> Non-Posix > does potentially have a place in .c files. It > depends. = >>Basically=2C as I see things=2C a small amount of #ifdef is ok. A > lot of= >> #ifdef becomes hard to read=2C and at some point=2C you split the files > in= >>to separate files=2C "no" #ifdefs -- or conceptually each file > can be viewe= >>d as wrapped in an #ifdef or other platform detection. You need > #ifdef i= >>n any =3BC that uses 64 bit integers=2C for example. Current > Microsoft&= >>nbsp=3Bcompilers do support "long long" but __int64 supporg goes > way back= >>=2C and __int64 is a better name anyway. >>What is the 128 bit integer type to be called -- "__int128" or > "long long l= >>ong". >> =3B >> =3B >>The names "char"=2C > "short"=2C "int"=2C > "long"=2C =3B"long long" = >>=3Bare mostly just > funny mistaken names. >>The right names to provide are explicitly sized=2C > int8=2C =3Buint8=2C = >>int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > = >>types such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >> =3B >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >> =3B >> =3B =3B- > Jay >> >> ________________________________ > id=3DstopSpelling> >> >> Date: Sat=2C 20 Dec 2008 00:09:41 > +0000 From: > dabenavidesd at yahoo.es<= >>BR>To: m3devel at elegosoft.com Subject: [M3devel] Rv: Re: > serial/Cygwin wa= >>s Re: [M3commit] FilePosixC.c >> >> >> >> Dear all: I'm glad that Jay and all the > people around a= >>re making contributions=2C so we can claim that as some day there were a > go= >>od number of supported platforms nowadays we also have. However I > have a= >> question=2C but its more a suggestion=2C the modification Tony is > referrin= >>g is not as standard as we would like since the type safety is affected > wit= >>h the change introduced allowing > <=3B> class=3DEC_current-rev>*>>>EXTERNAL> class=3DEC_current-rev>*>=3B > (untrusted) functions = >>in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 > (IMHO= >> it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C > = >>however doing that I think some clients could not =3B claim any > more th= >>ey are safe so that "fix" is not feasible. Can you push > to a lower level= >> (as in the UNSAFE FilePosix implementation module before committing the > ch= >>ange) or make it available in other Interface already available=2C why do > w= >>e need that change anyway? =3B Also IMHO we should leave out what > is no= >>t POSIX (we should not care for WIN64 although I know is needed by the C > co= >>mpiler preprocessor) out of that file. Although I don't have > the > formal = >>specification or program=2C but I can remember libm3 code was > specified > and= >> mechanically checked with the Extended Static Checker Modula-3 (which is > n= >>ot sound also=2C that is could be cheated=2C see > http://www.researchchannel= >>.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B > that wo= >>uld be a good reason for keeping as safe as possible and close to > historica= >>l interfaces and just rework implementations (even if they are not safe) > as= >> needed. I know that implementation is not safe but at least having it > well= >> enough implemented the ESC/M3 checker and the compiler would tell us the > l= >>ibrary should work Ok and is safe and (hopefully) without the errors > detect= >>ed by the ESC as we care for the details in the UNSAFE > implementation. T= >>here are any special interfaces we should have as "standard" > in > the lower l= >>evel runtime (hard enough question I know)=2C can we give > some hints/ideas > = >>on that? Thanks in > advance --- El vie=2C > 19/12/08=2C J= >>ay <=3Bjay.krell at cornell.edu>=3B > escribi=F3: >> > 5px">De: Jay <=3Bj= >>ay.krell at cornell.edu>=3B Asunto: Re: [M3devel] > serial/Cygwin Para: = >>"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C > "Tony" <=3Bhosking at cs.purd= >>ue.edu>=3B Fecha: viernes=2C 19 diciembre=2C 2008 > 1:25 >= >>=3B And I'll fix SOLsun/gnu. I /think/ it is ok now > -- changing my "= >>ino_t" to "m3_ino_t". I'm still > checking though. And I only > checke= >>d SOLgnu. I will check SOLsun. - > Jay > >>TD> >> = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- > > From martinbishop at bellsouth.net Sun Dec 28 23:14:14 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sun, 28 Dec 2008 16:14:14 -0600 Subject: [M3devel] Arbitrary precision integers Message-ID: <4957FA36.8080308@bellsouth.net> Does Modula-3 (or one of it's implementations) support arbitrary precision integers? From dabenavidesd at yahoo.es Sun Dec 28 23:48:35 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Dec 2008 22:48:35 +0000 (GMT) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> Message-ID: <460649.13613.qm@web28612.mail.ukl.yahoo.com> Hi, I guess BigInteger is a big integer of arbitrary size, you can look on http://www.opencm3.net/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html Also there is another effort made in a? B Sc's Thesis: http://www.emis.de/journals/NSJOM/Papers/30_3/NSJOM_30_3_123_130.pdf may be helpful, I haven't contact them, it should be helpful to ask. ? --- El dom, 28/12/08, Martin Bishop wrote: De: Martin Bishop Asunto: [M3devel] Arbitrary precision integers Para: m3devel at elegosoft.com Fecha: domingo, 28 diciembre, 2008 5:14 Does Modula-3 (or one of it's implementations) support arbitrary precision integers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 28 23:50:37 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:50:37 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081228100334.C41F3B04048@birch.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> Message-ID: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> This change makes me uneasy -- do you mean to say we have a uniform definition of sem_t when the different targets have different definitions? :-O On 28 Dec 2008, at 11:03, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/28 11:03:34 > > Modified files: > cm3/m3-libs/m3core/src/unix/Common/: m3makefile > cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile > cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile > Added files: > cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 > Removed files: > cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 > cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 > > Log message: > another common header file -- however this bought by defining sem_t > to have the maximum size and alignment of any system; > Solaris's 48 byte 64bit aligned version > The CM3 code base has one static sem_t in the pthread code. From hosking at cs.purdue.edu Sun Dec 28 23:46:44 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:46:44 +1100 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> The language itself does not support such things, though there may be a library implementing them somewhere. On 29 Dec 2008, at 09:14, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? From jay.krell at cornell.edu Mon Dec 29 00:49:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:49:25 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. There is only one sem_t in the system. It is global. This change just makes it usually larger and more aligned than necessary. The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. Usysdep.i3 seems best? - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jkrell at elego.de > Date: Mon, 29 Dec 2008 09:50:37 +1100 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > This change makes me uneasy -- do you mean to say we have a uniform > definition of sem_t when the different targets have different > definitions? :-O > > On 28 Dec 2008, at 11:03, Jay Krell wrote: > >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/28 11:03:34 >> >> Modified files: >> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >> Added files: >> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >> Removed files: >> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >> >> Log message: >> another common header file -- however this bought by defining sem_t >> to have the maximum size and alignment of any system; >> Solaris's 48 byte 64bit aligned version >> The CM3 code base has one static sem_t in the pthread code. > From jay.krell at cornell.edu Mon Dec 29 00:59:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:59:58 +0000 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> References: <4957FA36.8080308@bellsouth.net> <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> Message-ID: > though there may be > a library implementing them somewhere. The compiler itself has three such libraries embedded within it. :) Depending on how you define things. The front end cm3 does at least 64 bit integer math via 16 bit integers. cm3cg uses gmp and mpfr. The goals are probably the same for all three -- evaluate constant expressions correctly for the target. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: martinbishop at bellsouth.net > Date: Mon, 29 Dec 2008 09:46:44 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Arbitrary precision integers > > The language itself does not support such things, though there may be > a library implementing them somewhere. > > On 29 Dec 2008, at 09:14, Martin Bishop wrote: > >> Does Modula-3 (or one of it's implementations) support arbitrary >> precision integers? > From jay.krell at cornell.edu Mon Dec 29 00:55:16 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:55:16 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Note that I've been avoiding touching "mainstream" or already existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. Perhaps I should go ahead and cross that line, as my testability/testing allows? Slightly linked question is if user threads should remain. They don't work, I believe on any platform, but I understand could easily be made to. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; jkrell at elego.de > Date: Sun, 28 Dec 2008 23:49:25 +0000 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > > This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. > There is only one sem_t in the system. > It is global. This change just makes it usually larger and more aligned than necessary. > > > The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. > > > In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. > Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. > > > Usysdep.i3 seems best? > > > - Jay > > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jkrell at elego.de >> Date: Mon, 29 Dec 2008 09:50:37 +1100 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> This change makes me uneasy -- do you mean to say we have a uniform >> definition of sem_t when the different targets have different >> definitions? :-O >> >> On 28 Dec 2008, at 11:03, Jay Krell wrote: >> >>> CVSROOT: /usr/cvs >>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>> >>> Modified files: >>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>> Added files: >>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>> Removed files: >>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>> >>> Log message: >>> another common header file -- however this bought by defining sem_t >>> to have the maximum size and alignment of any system; >>> Solaris's 48 byte 64bit aligned version >>> The CM3 code base has one static sem_t in the pthread code. >> From hosking at cs.purdue.edu Mon Dec 29 01:14:02 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 11:14:02 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: User threads used to work on Solaris (since it uses getcontext/ setcontext to do thread switching instead of setjmp/longjmp) and these should work similarly on Linux/BSD/OSX. The only problem is that we need to use makecontext on some platforms to get a proper basis context for new threads (as opposed to using the hackier code that is there in ThreadPosix.m3. On 29 Dec 2008, at 10:55, Jay wrote: > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my testability/ > testing allows? > > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; jkrell at elego.de >> Date: Sun, 28 Dec 2008 23:49:25 +0000 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> >> This one I can undo and still feel ok -- just push sem_t into >> Utypes.i3 and expose it from a common Usem.i3. >> There is only one sem_t in the system. >> It is global. This change just makes it usually larger and more >> aligned than necessary. >> >> >> The danger would be if sem_t is ever embedded in something else OS- >> defined, or if the OS ever returned a pointer to one that Modula-3 >> code copied. >> >> >> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >> system-dependent stuff. >> Even existant Utypes.i3 have much redundancy -- all the ulong/ >> ushort/int8/int16/int32/int64 types are always the same, just stuff >> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >> should always be INT64 or UINT64. >> >> >> Usysdep.i3 seems best? >> >> >> - Jay >> >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jkrell at elego.de >>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> This change makes me uneasy -- do you mean to say we have a uniform >>> definition of sem_t when the different targets have different >>> definitions? :-O >>> >>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>> >>>> CVSROOT: /usr/cvs >>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>> >>>> Modified files: >>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>> Added files: >>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>> Removed files: >>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>> >>>> Log message: >>>> another common header file -- however this bought by defining sem_t >>>> to have the maximum size and alignment of any system; >>>> Solaris's 48 byte 64bit aligned version >>>> The CM3 code base has one static sem_t in the pthread code. >>> From jay.krell at cornell.edu Mon Dec 29 01:21:17 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 00:21:17 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Understood. You (someone) also has to fix the initialization. I think when you (you) moved the various user thread globals (?), such as the handler chain, into one nice thread object, "switchable" via just one pointer, it broke. I didn't look at the change in depth, I'm making a leap in my description of what you did. It is a /very/ reasonable design (or even likely The Correct Design), to collect up all the globals that represent a "thread" into one heap allocated struct...except it is an "object". I don't have SOLgnu/sun handy, this week, but can try it next week. I expect it fails as PPC_LINUX was. I THINK the "object" needs to be changed to a "traced ref" and then it can be initialized using, maybe, a preallocated global, for the first thread, or at least using regular "Malloc" and not going through the garbage collection...but I'm really just not sure. If I get X86_MSDOS/DJGPP up and going, I'll be a bit more motivated..but I think I'll so I386_SOLARIS first, since it is perhaps more modern/mainstream. You know, I tried having it NEW the object on-demand, but I think that ended up recursing to about the same place -- PushEFrame. It's "difficult" to repro now for me, this week, based on hardware/OS availability,because I ripped out PPC_LINUX user thread support (unless I install NetBSD/FreeBSD in a VM or use birch, all of which are, honestly, easy). - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 11:14:02 +1100 > CC: jkrell at elego.de; m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > User threads used to work on Solaris (since it uses getcontext/ > setcontext to do thread switching instead of setjmp/longjmp) and these > should work similarly on Linux/BSD/OSX. The only problem is that we > need to use makecontext on some platforms to get a proper basis > context for new threads (as opposed to using the hackier code that is > there in ThreadPosix.m3. > > On 29 Dec 2008, at 10:55, Jay wrote: > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? >> >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. >> >> - Jay >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: hosking at cs.purdue.edu; jkrell at elego.de >>> Date: Sun, 28 Dec 2008 23:49:25 +0000 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> >>> This one I can undo and still feel ok -- just push sem_t into >>> Utypes.i3 and expose it from a common Usem.i3. >>> There is only one sem_t in the system. >>> It is global. This change just makes it usually larger and more >>> aligned than necessary. >>> >>> >>> The danger would be if sem_t is ever embedded in something else OS- >>> defined, or if the OS ever returned a pointer to one that Modula-3 >>> code copied. >>> >>> >>> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >>> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >>> system-dependent stuff. >>> Even existant Utypes.i3 have much redundancy -- all the ulong/ >>> ushort/int8/int16/int32/int64 types are always the same, just stuff >>> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >>> should always be INT64 or UINT64. >>> >>> >>> Usysdep.i3 seems best? >>> >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> From: hosking at cs.purdue.edu >>>> To: jkrell at elego.de >>>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>>> Subject: Re: [M3commit] CVS Update: cm3 >>>> >>>> This change makes me uneasy -- do you mean to say we have a uniform >>>> definition of sem_t when the different targets have different >>>> definitions? :-O >>>> >>>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>>> >>>>> CVSROOT: /usr/cvs >>>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>>> >>>>> Modified files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>>> Added files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>>> Removed files: >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>>> >>>>> Log message: >>>>> another common header file -- however this bought by defining sem_t >>>>> to have the maximum size and alignment of any system; >>>>> Solaris's 48 byte 64bit aligned version >>>>> The CM3 code base has one static sem_t in the pthread code. >>>> > From lemming at henning-thielemann.de Mon Dec 29 01:42:24 2008 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 29 Dec 2008 01:42:24 +0100 (CET) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: On Sun, 28 Dec 2008, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? http://modula3.elegosoft.com/cm3/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html It's not well optimized and maybe not enough tested. If you find it too slow, you might write wrapper to the GNU multiprecision library GMP. Maybe SWIG can help writing the wrapper. (or you try Haskell, which also uses GMP :-) From wagner at elegosoft.com Mon Dec 29 08:46:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 29 Dec 2008 08:46:55 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Quoting Jay : > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my > testability/testing allows? I wanted to comment on this before. I've no objection at trying to remove duplicated code as far as possible, but when changing system interfaces (Uxxx.is), please make sure that they really work on the target platform in question; if possible, add some tests. > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. I was not aware that user threads are broken; why? Anyway, this code should not be removed; I was rather hoping for a compile-time option to switch between user and system threads in cm3. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Mon Dec 29 09:16:13 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 19:16:13 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Message-ID: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > Quoting Jay : > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? > > I wanted to comment on this before. I've no objection at trying to > remove duplicated code as far as possible, but when changing system > interfaces (Uxxx.is), please make sure that they really work on the > target platform in question; if possible, add some tests. > >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. > > I was not aware that user threads are broken; why? > Anyway, this code should not be removed; I was rather hoping for a > compile-time option to switch between user and system threads in cm3. Definitely doable! I've been meaning to do this for a long time now. > > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From jay.krell at cornell.edu Mon Dec 29 09:47:02 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 08:47:02 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> Message-ID: cm3 -DNOPTHREAD compiles m3core for user threads. I /sort of/ (almost) haven't removed anything. I did remove PPC_LINUX stuff. In new platforms (openbsd, {amd64,sparc}_linux, cygwin), I might have initially added partial support for user threads, never complete, tested, working support, and then removed it. User threads are broken because PushEFrame uses uninitialized NULL self. PushEFrame is always called before ThreadF__Init (which initializes self), but the other two threading libraries (which don't have "self" at all) initialize on demand what they need in PushEFrame (the thread local handler stack). I think self/Thread.T needs to be a traced ref, instead of an object, so the initial value can be to a preallocated static/global. I'm not sure of the language here. "object" seems to deprive you of the "value type". Whereas refs do not. I tried allocating self on demand in PushEFrame but that didn't work. It ended up back in PushEFrame. I tried more tactics after that, but then gave up. Or move the handle stack out of Thread.T and into a separate global like it was. Either change would not affect pthreads or NT threads. I would like to see how much I can whittle down m3core of platform-specific stuff, and then maybe see about adding user thread support for the platforms that never had it. I'll add I386_SOLARIS soon, and don't want to duplicate a "large" directory of files as part of that. I really dislike the multiplicity of such a system. I have another chunk of m3core about to go though, for systems that don't have user threads. Maybe not one that is worthwhile. I'm still not much of a believer in user threads. Essentially I have always had kernel threads, with a processor-portable interface, and one machine-wide scheduler..seems good.. Every thread mechanism needs its own locking mechanism, which won't interace with any other thread or locking mechanism. Kind of like picking a C compiler or library -- you want there to be just one obvious choice per-platform and have everyone use it, so that there is interop and better overall scheduling. You know..Modula-3 I think has problems when the executable doesn't use the Modula-3 runtime and/or when threads are created outside the runtime, at then run Modula-3 code. That seems unfortunate. I /should/ be able to write a "plugin .dll" in Modula-3 and have it loaded by a C or C++ application, and have garbage collection work. Win32 is also kind of wonky here, in that it has "fibers". Fibers are like swapcontext, plus the stack allocation is handled for you. Anyone using them has to implement their own scheduler and their own locking mechanisms. Win32 critical sections, mutexes, etc. don't interact "correctly" with them. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: wagner at elegosoft.com > Date: Mon, 29 Dec 2008 19:16:13 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > >> Quoting Jay : >> >>> >>> Note that I've been avoiding touching "mainstream" or already >>> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >>> >>> Perhaps I should go ahead and cross that line, as my testability/ >>> testing allows? >> >> I wanted to comment on this before. I've no objection at trying to >> remove duplicated code as far as possible, but when changing system >> interfaces (Uxxx.is), please make sure that they really work on the >> target platform in question; if possible, add some tests. >> >>> Slightly linked question is if user threads should remain. >>> They don't work, I believe on any platform, but I understand could >>> easily be made to. >> >> I was not aware that user threads are broken; why? >> Anyway, this code should not be removed; I was rather hoping for a >> compile-time option to switch between user and system threads in cm3. > > Definitely doable! I've been meaning to do this for a long time now. > >> >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, >> Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 >> 45 86 95 >> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: >> Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: >> DE163214194 >> > From mika at async.caltech.edu Mon Dec 29 10:23:17 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 29 Dec 2008 01:23:17 -0800 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." Message-ID: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Jay writes: > ... > >I'm still not much of a believer in user threads. >Essentially I have always had kernel threads, with a processor-portable >interface, and one machine-wide scheduler..seems good.. > ... Except that on some machines, some operating systems, Modula-3's user threads are much more efficient (and if not, far more easily modifiable) than the provided kernel threads when you perform certain types of operations in certain situations. I often write programs with many (hundreds, thousands) of active threads and run into these sorts of problems frequently. You could argue that it should not be so... that every OS should provide kernel threads that are better than Modula-3's own user threads under all conceivable workloads, but what good does that do? Seems better to have a way of avoiding that can of worms by being able to control one's own implementation. Another reason that occurred to me when reading some very old M3 documentation is that there do occasionally exist systems that don't have a real operating system. There may also be other deficient systems that don't have pthreads. With an existing user-level threading library you can easily support such systems. (M3 has supported such systems in the past...) Mika From jay.krell at cornell.edu Mon Dec 29 10:47:26 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 09:47:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> References: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Message-ID: I'm aware of systems without threads, e.g. MS-DOS or apparently many of the systems Modula-3 first ran on. (Really? pthreads wasn't very available back then?) The case of many threads, um.. You don't run out of address space on the stacks? I guess the Modula-3 stuff honors requests for very small stacks? (need to fix that it leaks a pthread_attr here..) Not a problem on 64bit systems. The large number of threads doesn't bog down scheduling? Is it hundreds/thousands of threads alive at the same time, or serially? Serially, no big deal. Modifiable, sure. The more you implement "yourself", the more you can change. You can hope for folks to build in some "configurability" and "tunability", but nothing beats the ability to change or replace the thing arbitrarily. Maybe having hundreds/thousands is "wrong"? I have repeatedly seen guidance against creating "many" threads. Better to have a "thread pool", and/or use "completion ports" (aka select?). Maybe you should have just a few threads, and then decide for yourself what each should run? You know, isn't the guidance to have roughly one thread per processor? You aren't going to scale beyond that..unless you dedicate a thread per slow i/o, which is wasteful. You know there is this whole thing where..too many cooks spoil the broth. Having multiple schedulers on the system means they won't do an ideal job. It is best to have one scheduler, that "knows everything" -- knows about all the threads and what they are waiting for. The cost of threads..is both low and high. - Jay ---------------------------------------- > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 01:23:17 -0800 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > Jay writes: >> > ... >> >>I'm still not much of a believer in user threads. >>Essentially I have always had kernel threads, with a processor-portable >>interface, and one machine-wide scheduler..seems good.. >> > ... > > Except that on some machines, some operating systems, Modula-3's > user threads are much more efficient (and if not, far more easily > modifiable) than the provided kernel threads when you perform certain > types of operations in certain situations. I often write programs > with many (hundreds, thousands) of active threads and run into these > sorts of problems frequently. > > You could argue that it should not be so... that every OS should > provide kernel threads that are better than Modula-3's own user > threads under all conceivable workloads, but what good does that > do? Seems better to have a way of avoiding that can of worms by > being able to control one's own implementation. > > Another reason that occurred to me when reading some very old > M3 documentation is that there do occasionally exist systems > that don't have a real operating system. There may also be > other deficient systems that don't have pthreads. With an existing > user-level threading library you can easily support such systems. > (M3 has supported such systems in the past...) > > Mika From jay.krell at cornell.edu Tue Dec 30 03:07:24 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:07:24 +0000 Subject: [M3devel] current tinderbox status, broken by me? Message-ID: NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho. - Jay From stsp at elego.de Tue Dec 30 03:16:46 2008 From: stsp at elego.de (Stefan Sperling) Date: Tue, 30 Dec 2008 03:16:46 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229203344.692E1B04044@birch.elegosoft.com> References: <20081229203344.692E1B04044@birch.elegosoft.com> Message-ID: <20081230021646.GA8558@jack.stsp.name> On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/29 21:33:44 Hey Jay, > Log message: > enough I386_OPENBSD support to build a bootstrap package, > that builds on the target, and results in a cm3 that > runs and correct errors out for lack of cm3.cfg, Cool, I'd like to play with this, once you have a bootstrap binary. Stefan From jay.krell at cornell.edu Tue Dec 30 03:28:09 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:28:09 +0000 Subject: [M3devel] regular cross builds, package format? Message-ID: So..it is reasonably easy to do a cross build, that outputs a bunch of assembly files, some C files, a "script" or makefile to build them. What you (I) do with such output is copy it to the target machine and run the script, which requires a C compiler, linker, and whatever libraries, and whatever "script runner" (sh, make, etc.). I've been using this stuff to bring up new platforms. It works well enough. There are a number of obvious questions. Should the tinderbox go ahead and do that, for "every" target platform? ("every" defined as known working ones) What should the form of the "cross build archive" be? Or, for that matter, what should be form of "native distributions"? Should it be possible to build one "intermediate" distribution that can be "finished" on either any autoconf-able Posix platform, or any supported platform? That is, should the platform-specificity be factored out of the intermediate representation, and the cross or universal or even native (unlikely) format be that of portable intermediate files, and the "finish" step includes building the backend (not fast). I gather this last thing was done by PM3, though I also gather it isn't a small task. (I need to look at their code.) Platform-specifity leaks into the intermediate representation presumably a lot. You know, the size of integers, alignment affects on layout, etc. As well, if some systems use RTOSmmap.c and others RTOSbr.c -- or even .m3 files, the "universal" distribution would have to carry both files and delay the choice. If you reject the "universal" notion, there remain some questions, though less complicated ones. Simpple ones like what the "finish script" is. I generate a makefile, but it seems to be buggy. Sometimes there is no incrementality, it builds everything every time. Sometimes it generates a warning for each file. I don't know why, but since these are so far only for my use, ok. Of course there remains the question of cminstall or no cminstall. I've been getting by without cminstall for quite some time across a few platforms. A "universal" distribution would probably chose neither of the current options, but something that works better like autoconf. I'm not a huge fan of autoconf, but it has some very tempting characteristics, where it can sniff out system behavior as long as it is either in a known set or not too far off. - Jay From jay.krell at cornell.edu Tue Dec 30 03:57:12 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:57:12 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081230021646.GA8558@jack.stsp.name> References: <20081229203344.692E1B04044@birch.elegosoft.com> <20081230021646.GA8558@jack.stsp.name> Message-ID: Cool. You can try http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-I386_OPENBSD-1.tar.gz, but hopefully soon I'll have "std" and "min". The boot archive builds cm3 only, no libs, no cm3cg -- though it should be enough to build them natively (plus a current source tree). If you use this, be aware that something is a little off in my bootstrapping process in that none of do-cm3-std do-cm3-min upgrade etc., seem quite right. upgrade in particular requires pkg/m3core and pkg/libm3. None of the sets imply TO ME building both m3cc and cm3, though "front" might actually (it ought not build m3cc). Also, the sorting to build in dependency order in pylib.py seems to not be working, strange. Anyway, this is fairly minor. There is also a definite possiblity of cross-building a complete distribution, you just have to build GNU cc (similar to building m3cg), GNU as, GNU ld, and either copy or build the C runtime and startup files...at least for *_*BSD, *_LINUX, SOLgnu, I386_CYGWIN, I don't know about the others (Darwin does not use GNU ld nor I think GNU as, whatever it does use, I don't know if it is easily re-hosted, similar SOLsun.) (Tony, please speak up if you really meant X86_* and not I386_*.) - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 03:16:46 +0100 > From: stsp at elego.de > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/29 21:33:44 > > Hey Jay, > >> Log message: >> enough I386_OPENBSD support to build a bootstrap package, >> that builds on the target, and results in a cm3 that >> runs and correct errors out for lack of cm3.cfg, > > Cool, I'd like to play with this, once you have a bootstrap binary. > > Stefan From dabenavidesd at yahoo.es Tue Dec 30 07:01:45 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 30 Dec 2008 06:01:45 +0000 (GMT) Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: Message-ID: <908914.75062.qm@web28601.mail.ukl.yahoo.com> In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? I would like to hear your comments on that, let me know what else is missing to know the FilePosix is safe. Thanks in advance, --- El lun, 29/12/08, Jay wrote: De: Jay Asunto: [M3devel] current tinderbox status, broken by me? Para: "m3devel" Fecha: lunes, 29 diciembre, 2008 9:07 NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 30 08:48:17 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:48:17 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <908914.75062.qm@web28601.mail.ukl.yahoo.com> References: <908914.75062.qm@web28601.mail.ukl.yahoo.com> Message-ID: The C code I introduced should behave identically to or better than the Modula-3 code it replaced. The Modula-3 code dependend on tedious error-prone rewriting of C headers in Modula-3. Mistakes there easily lead to the overall safety of the system being violated. I believe the C code adheres to the safety guarantee below. Can you check? - Jay ________________________________ > Date: Tue, 30 Dec 2008 06:01:45 +0000 > From: dabenavidesd at yahoo.es > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] current tinderbox status, broken by me? > > In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." > Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? > For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? > I would like to > hear your comments on that, let me know what else is missing to know the FilePosix is safe. > Thanks in advance, > > --- El lun, 29/12/08, Jay wrote: > De: Jay > Asunto: [M3devel] current tinderbox status, broken by me? > Para: "m3devel" > Fecha: lunes, 29 diciembre, 2008 9:07 > > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix..m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 > -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to clashing > definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I > had another new platform defined in one or another file and then when I > backed down and just did one reasonly finished platform, I forgot at first to > cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to run > into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, additions > are safe" annotation? > Probably not. It's probably too gray and more complicated than that, since > additions in general are not as safe as people thing. Adding to a record -- > folks might be copying based on the size and might get back an older smaller > version. Adding to an enum, folks might be switching on it and expect to have > the range covered. Adding a function to an interface though, probably ok. Not > the case here, and perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an > enum, and such annotated > types, switches on them must have an else clause, and they must not be used for > array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf is > recorded in both Csetjmp.i3 and Target..i3. Duplication is generally bad imho > > From jay.krell at cornell.edu Tue Dec 30 08:51:56 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:51:56 +0000 Subject: [M3devel] darwin-generic waitpid? Message-ID: It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and having a C wrapper do the translation. As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. - Jay From jay.krell at cornell.edu Tue Dec 30 09:34:03 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 08:34:03 +0000 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the waitpid(nohang)/sleep() loop that I had removed in other places, alas -- replaced in pthreads by waitpid(hang), whereas userthreads still has waitpid(nohang)/sleep (another flaw in user threads?) and..isn't this a little odd: RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); If there are no users of this, I'll just comment it out. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; m3devel at elegosoft.com > Date: Tue, 30 Dec 2008 07:51:56 +0000 > Subject: [M3devel] darwin-generic waitpid? > > > It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and > having a C wrapper do the translation. > > > As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. > > > - Jay From wagner at elegosoft.com Tue Dec 30 11:01:24 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 30 Dec 2008 11:01:24 +0100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 for details. It occurs whenever you add or remove target platforms. It is exactly what the last-ok build was made for :-) As for your other comments, I would appreciate if the compiler would output the exact conflicting file paths, but I believe that the information is not easily available at that point. Olaf Quoting Jay : > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 30 11:04:23 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:04:23 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> References: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> Message-ID: Ah, I thought the Tinderbox always ran upgrade.sh first. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 11:01:24 +0100 > From: wagner at elegosoft.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] current tinderbox status, broken by me? > > This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 > for details. It occurs whenever you add or remove target platforms. > > It is exactly what the last-ok build was made for :-) > > As for your other comments, I would appreciate if the compiler would > output the exact conflicting file paths, but I believe that the > information is not easily available at that point. > > Olaf > > Quoting Jay : > >> >> NEXT Fatal Error: bad version stamps: SocketPosix.m3 >> 4678 >> 4679 version stamp mismatch: Compiler.Platform >> 4680 => SocketPosix.m3 >> 4681 => Compiler.i3 >> 4682 version stamp mismatch: Compiler.ThisPlatform >> 4683 => SocketPosix.m3 >> 4684 => Compiler.i3 >> 4685 NEXT *** execution of cm3 -build >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? failed *** >> 4686 compile return value: 0 >> 4687 [end compile 2008.12.30 02:25:17] >> 4688 *** COMPILE FAILED >> No More Errors >> >> >> 1) >> Is it reasonable to suggest that the compiler output the source to >> clashing definitions? >> Or is that a bogus naive suggestion? >> It is probably legitimate, at least to identify the files in question. >> Could be that not enough information is around to identify "what >> changed", just some "summary hash". >> >> >> 2) Is it really broken? >> I did go slightly round and round on this. >> I had another new platform defined in one or another file and then >> when I backed down and just did one reasonly finished platform, I >> forgot at first to cleanup all the debris. >> But it certainly worked when I commited. >> I'm progressing on the native build, so if it is broken, I expect to >> run into it myself. >> >> >> 3) Is it reasonable to annotate a type with some sort of "relax, >> additions are safe" annotation? >> Probably not. It's probably too gray and more complicated than that, >> since additions in general are not as safe as people thing. Adding >> to a record -- folks might be copying based on the size and might >> get back an older smaller version. Adding to an enum, folks might be >> switching on it and expect to have the range covered. Adding a >> function to an interface though, probably ok. Not the case here, and >> perhaps not something the compiler even complains about. >> One that that might be eek-out-able is adding to an enum, and such >> annotated types, switches on them must have an else clause, and they >> must not be used for array indexing? >> Eh, maybe just leave it alone. >> >> >> Anything else? Is this just basically how it has to be? >> OR, how about, can the duplicity be eliminated? >> Like, can Compiler.i3 be fully generated by the compiler? >> Similarly, Csetjmp.i3 should be generated by the compiler, eh? >> Or at least partly? You know, the size and alignment of the jmp_buf >> is recorded in both Csetjmp.i3 and Target.i3. Duplication is >> generally bad imho > > > > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > From jay.krell at cornell.edu Tue Dec 30 11:08:53 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:08:53 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: I wish: 1) you could put declarations in .m3 files 2) could be entirely optional, as long as the C functions are named Module__Function. There are three implementations of RTOS.GetMemory. In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 08:00:29 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/30 08:00:29 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > FilePosixC.c m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 > > Log message: > put the C functions in their own interface, and other small change > (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) > From jay.krell at cornell.edu Tue Dec 30 18:38:33 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 17:38:33 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: [was truncated, and now edited slightly] > I wish: > > > 1) you could put declarations in .m3 files > > > 2) could be entirely optional, as long as the C functions are named Module__Function. > > > There are three implementations of RTOS.GetMemory. > In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. > > > - Jay > > ---------------------------------------- >> Date: Tue, 30 Dec 2008 08:00:29 +0000 >> To: m3commit at elegosoft.com >> From: jkrell at elego.de >> Subject: [M3commit] CVS Update: cm3 >> >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/30 08:00:29 >> >> Modified files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 >> FilePosixC.c m3makefile >> Added files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 >> >> Log message: >> put the C functions in their own interface, and other small change >> (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) >> From hosking at cs.purdue.edu Wed Dec 31 00:47:46 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:47:46 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: Message-ID: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay From hosking at cs.purdue.edu Wed Dec 31 00:48:43 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:48:43 +1100 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: Yes, this definitely needs cleanup. On 30 Dec 2008, at 19:34, Jay wrote: > > On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the > waitpid(nohang)/sleep() loop that I had removed in other places, > alas -- replaced in pthreads by waitpid(hang), whereas userthreads > still has waitpid(nohang)/sleep (another flaw in user threads?) > > and..isn't this a little odd: > > RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); > > If there are no users of this, I'll just comment it out. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; m3devel at elegosoft.com >> Date: Tue, 30 Dec 2008 07:51:56 +0000 >> Subject: [M3devel] darwin-generic waitpid? >> >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. >> >> >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay From hosking at cs.purdue.edu Wed Dec 31 00:50:35 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:50:35 +1100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: These result from inconsistency between the platforms defined statically in cm3 and the platforms declared in m3core. You need to use the version in m3core that matches what you had when building cm3. Then, extend cm3 before compiling the new m3core version. On 30 Dec 2008, at 13:07, Jay wrote: > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3- > ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/ > birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho From jay.krell at cornell.edu Wed Dec 31 01:09:58 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 00:09:58 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: I'd like to avoid cloning any headers, and use clear portable C wrappers to bridge between Modula-3 and stuff written/declared in C. e.g. no bit fields. These are contrary goals. Or, at least, only clone headers that are essentially identical across all Posix platforms, or which the only variables are endianness and word size. There are a few possible compromises. Currently there is linux-common, openbsd-common. I would NOT like to add netbsd-common, freebsd-common, solaris-common. As such, I'm pushing stuff from "*-common" to plain "Common". As well, by not cloning header content that is unused in the cm3 tree, much is saved. Compare the number of lines in LINUXLIBC6 vs. linux-common. But I'd like to get further than that. Similarly, not cloning header content that doesn't present a portable interface, which tends to lead to it not being used, since most of the cm3 code is factored only on Posix vs. Win32, and not one Posix superset or another. I also believe that C is "everywhere" and will remain "everywhere". Except very constrained "safe" environments like a web browser or Java-only Google phone, where Modula-3 is about as far as being able to run as C is. I think C wrappers are very preferable to cloning headers. The endian-ness is actually abstracted out in normal C code. There's the various macros for cracking the 32 bit status. Ok, maybe that is apples and oranges. It is abstracted out in the client C code and the client Modula-3 code, except that Modula-3 client code uses LOOPHOLE. Both the C headers and Modula-3 headers are "messy", the C headers sometimes more so, such as to cater to other (BSD) interfaces. I realize there are the Win32 headers to be concerned with as well. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 10:47:46 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > On 30 Dec 2008, at 18:51, Jay wrote: > >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. > > I am concerned that so much C wrapper stuff is starting to creep in. > I want to be able to see the endianness in the Modula-3 code, not have > it squirrelled away in C code. What if I don't have a C compiler? > Jay, please be careful with your assumptions about platforms always > having C. > >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay > From hosking at cs.purdue.edu Wed Dec 31 01:47:48 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 11:47:48 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Sure, I understand. I think what you are doing is great for improving the reliability of the C-interface code. Are you able to test all the platforms? I am worried we are going to have bugs introduced that someone (you!) will need to chase down. On 31 Dec 2008, at 11:09, Jay wrote: > > I'd like to avoid cloning any headers, and use clear portable C > wrappers to bridge between Modula-3 and stuff written/declared in C. > e.g. no bit fields. > > These are contrary goals. > > Or, at least, only clone headers that are essentially identical > across all Posix platforms, or which the only variables are > endianness and word size. > > There are a few possible compromises. > Currently there is linux-common, openbsd-common. > I would NOT like to add netbsd-common, freebsd-common, solaris-common. > As such, I'm pushing stuff from "*-common" to plain "Common". > > As well, by not cloning header content that is unused in the cm3 > tree, much is saved. > Compare the number of lines in LINUXLIBC6 vs. linux-common. > But I'd like to get further than that. > > Similarly, not cloning header content that doesn't present a > portable interface, which tends to lead to it not being used, since > most of the cm3 code is factored only on Posix vs. Win32, and not > one Posix superset or another. > > I also believe that C is "everywhere" and will remain "everywhere". > Except very constrained "safe" environments like a web browser or > Java-only Google phone, where Modula-3 is about as far as being able > to run as C is. > > I think C wrappers are very preferable to cloning headers. > > The endian-ness is actually abstracted out in normal C code. > There's the various macros for cracking the 32 bit status. > Ok, maybe that is apples and oranges. It is abstracted out in the > client C code and the client Modula-3 code, except that Modula-3 > client code uses LOOPHOLE. Both the C headers and Modula-3 headers > are "messy", the C headers sometimes more so, such as to cater to > other (BSD) interfaces. > > I realize there are the Win32 headers to be concerned with as well. > > - Jay > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jay.krell at cornell.edu >> Date: Wed, 31 Dec 2008 10:47:46 +1100 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] darwin-generic waitpid? >> >> On 30 Dec 2008, at 18:51, Jay wrote: >> >>> >>> It looks like waitpid's output is endian-dependent, and that >>> therefore darwin-generic is wrong on some architectures, asis linux- >>> common and openbsd-common. >>> >>> >>> I expect to fix this shortly, by replacing all the Modula-3 result >>> bitfields with uint8_t and >>> having a C wrapper do the translation. >> >> I am concerned that so much C wrapper stuff is starting to creep in. >> I want to be able to see the endianness in the Modula-3 code, not >> have >> it squirrelled away in C code. What if I don't have a C compiler? >> Jay, please be careful with your assumptions about platforms always >> having C. >> >>> As well, for increased portability, the flags will be input uint8_t >>> fields in the record, rather than platform-dependent flags. >>> >>> >>> - Jay >> From jay.krell at cornell.edu Wed Dec 31 02:25:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 01:25:31 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: I am not able to test everything this week, on vacation, just have two laptops with me, and not quite /everything/ at home yet, but I have available a lot now and am still working on getting and setting up more hardware and operating systems, and maybe will use virtualization more (I386_OPENBSD is in VirtualPC, Solaris/x86 in VirtualBox bugchecks the host too frequently(. I'm not sure the inventory is important or impressive or embarassing. :) I've got a variety of MIPS64 (SGI), PPC64 (RS/6000), PPC32 (Apple), ARM (Western Digital MyBook "World Edition), IA64 (HP Integrity), SPARC64 (Sun), AMD64 (Sun and PCs), running variety of Linux, OpenBSD, Darwin, AIX, not currently NetBSD but will, NT, Solaris.. Alphas are on the way, HP-PA might be on the way, the main missing piece is PPC64_DARWIN hardware. :) - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 05:32:25 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 04:32:25 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: Tony, Hi, sorry, I'm a little confused. Beware C wrappers -- don't write them. That's what it sounded like at first. Or: Beware C wrappers -- be careful writing them. ? On the second, well, yeah, ok. I haven't actually yet improved much -- since I stick to "new" ports. "Soon" I might take larger steps. I'll probably first cave and add user thread support to {PPC,AMD64,SPARC{32,64}}_LINUX, I386_SOLARIS, *_OPENBSD, etc., while continuing to minimize or eliminate header clones. And then maybe move LINUXLIBC6 to linux-common/common. I should say..um..IF your concern is just "be careful writing C code"... I wonder how far I should go. I'm obessive. I keep looking at linux-common/openbsd-common/cygwin and I wonder what I can remove. And how. Removing unused stuff, easy. Wrapping functions that actually are functions, not too bad. But types and constants..you can try to write functions to take over all their uses, but I don't think that will do for all of them. For example, struct flock I removed. The "worst" cases are probably: Uerrno Upthread In particular, these are the files for which C wrapping is likely to do the "most damage" with respect to efficiency and code structure. Uerrno..the way to wrap it in C is, well..one thing I do NOT want to do is go through and make up our own errno values and map back and forth everywhere. Yuck. However, one thing that seems half reasonable is to have GetEAGAIN(), GetEINVALID(), GetENOMEM(), etc. or a bulk struct { int EINVAL, ENOMEM, etc. } * GetErrnoValues(). The second version is "less chatty" but risky, like if the struct gets copied and changes size through time (versioning). The first is much safer. In both cases, the errors loses their const-ness and you can, I assume, no longer switch/case on them. Now, error checking..efficiency..probably ok -- code that is dealing with errors doesn't have to be fast. Change the switches/cases to if/elseif/elseif ladders. I think I will do this. Upthread, here I think part of the answer is that all of the functions have identical prototypes. So "half" the file just goes to Common and done, no C wrapping. That leaves the types. sem_t, which is vaguely related, there is just one of. I think we can just declare that variable in C, since you don't like the over allocation/alignment. But that's an unusual benefit -- something there is only one of. For types that are created repeatedly..well, it depends. I have to look into the allocator interface.. if the objects are heap allocated, one thing you can maybe do is like: RTAllocator(Upthread.GetSize_rwlock(), Upthread.GetAlign_rwlock()). Assuming size and alignment are parameters, or assuming that alignment is always maximal. If the objects are stack allocated, well, that breaks down. You /could/ heap allocate them always, but that's a sort of inefficiency I'm reluctant to introduce. I don't mind turning constants, esp. seldom used ones, into function calls, or copying small structs (stat), but turning stack allocation into heap allocation, yuck. Another answer is to leave these alone. Generate Uerrno.i3 from Uerrno.i3.cpp and leave it that. Maybe generate them as a matter of course in any /native/ build and compare to the "checked in" copy. Well, um, native builds also benefit from being able to run tests, so you can accomplish the same thing there -- pass the constants, type sizes, etc. to C code that asserts equivalence. Upthread.i3 could be similar. To address the types vs. prototypes, I'd say move the types, and probably all the initializers, into a per-system Upthreadtypes.i3, or Usysdep.i3. Usocket.i3 is similar -- identical prototypes, but constants vary. Move the constants into "Usysdep.i3"? Or use accessor functions? I have to look at the code more besides. I don't have a plan. Anyway, at the very least, we have: jay at ppca:/dev2/cm3/m3-libs/m3core/src/unix$ wc -l linux-common/*.i3 415 total openbsd-common => 516 wc -l linux-libc6/*.i3 4011 Not bad to be at 10%, and user threads won't regress all that much. Maybe stop there? Well, Uutsname.i3 is probably going away. It has just one user. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 06:33:22 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 05:33:22 +0000 Subject: [M3devel] variations of waitpid..? Message-ID: variations of waitpid.. Hey, we already have, that I put in on my initiative, months ago: C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) VAR result: int; statusM3: Uwaitpid.waitpid_status_t; CONST Delay = 0.1D0; BEGIN LOOP result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Pause(Delay); END; 0 *> RETURN statusM3.w_Loophole; END WaitProcess; pthreads and cygwin: C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 and C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) VAR statusM3: Uwaitpid.waitpid_status_t; BEGIN LOOP WITH r = Uwaitpid.waitpid(pid, statusM3) DO IF r> 0 THEN RETURN statusM3.w_Loophole; END; < 0*> END; END; END WaitProcess; win32: PROCEDURE WaitProcess (pid: int): int = BEGIN RETURN 0; END WaitProcess; ThreadPThread and WaitProcessCygwin should share code, but that's not the point. However, now, to fix sysutils, we need something LIKE: UNSAFE INTERFACE SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; BEGIN END SystemPosixWaitPid. pthreads: UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = BEGIN RETURN waitpid(pid, status, 0); END WaitPid; BEGIN END SystemPosixWaitPidEfficient. user threads: UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = VAR result: int; CONST Delay = 0.1D0; BEGIN LOOP result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Thread.Pause(Delay) END; END WaitPid; BEGIN END SystemPosixWaitPidPause. questions: Aside: Where do I put "unsafe"? Relevant: Sysutils has to implement this itself, right? Due to bootstrapping concerns. But we should still put /something/ in m3core, right? So then the big question, what to call it? Probably, the existing WaitProcess that does more than this WaitPid, should be scaled back and only do what WaitPid does? I will likely commit exactly what is shown above. The new interface and modules are in sysutils. The interface is private. The unfinished part is putting something in m3core, that sysutils could use, if not for boostrapping issuers, that it could use in the future. I kind of thing something a bit more is needed. You know, the Win32 implementation that asserts(false) is suspicious. At the very least, it should be removed. I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 interfaces? I also suspect part of the problem is that sysutils implements to a large extent the same abstractions as m3core and/or libm3, but with variations, causing it to be unable to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions maybe are not general/featureful enough. (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) ?? - Jay From jay.krell at cornell.edu Wed Dec 31 08:13:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 07:13:02 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: References: Message-ID: Hm. How about: PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? ? true for pthreads/win32/cygwin false for userthreads duplicated privately in sysutils to avoid depending on newer m3core like I did for the portable C-wrapped waitpid ? That way..the other code can do whatever it already is doing, no need to come up with abstractions for looping with nohang, vs. that plus the status loophole. The older fix I put it can be shrunk. The name of this is debatable, but I think the minimal level of abstraction is goodness. The name is actually perhaps too..complicated..like, what does it mean? Who uses it? What do they do with it? It means that waiting on a process to end allows threads in the waiting process to run. Perhaps it is too low level. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Wed, 31 Dec 2008 05:33:22 +0000 > Subject: [M3devel] variations of waitpid..? > > > variations of waitpid.. > > Hey, we already have, that I put in on my initiative, months ago: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) > VAR > result: int; > statusM3: Uwaitpid.waitpid_status_t; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); > IF result # 0 THEN EXIT END; > Pause(Delay); > END; > 0 *> > RETURN statusM3.w_Loophole; > END WaitProcess; > > pthreads and cygwin: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 > and > C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) > VAR > statusM3: Uwaitpid.waitpid_status_t; > BEGIN > LOOP > WITH r = Uwaitpid.waitpid(pid, statusM3) DO > IF r> 0 THEN > RETURN statusM3.w_Loophole; > END; > < 0*> > END; > > END; > END WaitProcess; > > win32: > PROCEDURE WaitProcess (pid: int): int = > BEGIN > > RETURN 0; > END WaitProcess; > > ThreadPThread and WaitProcessCygwin should share code, but that's not the point. > > However, now, to fix sysutils, we need something LIKE: > > UNSAFE INTERFACE SystemPosixWaitPid; > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; > > BEGIN > END SystemPosixWaitPid. > > pthreads: > > UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > BEGIN > RETURN waitpid(pid, status, 0); > END WaitPid; > > BEGIN > END SystemPosixWaitPidEfficient. > > user threads: > > UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > VAR result: int; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); > IF result # 0 THEN > EXIT > END; > Thread.Pause(Delay) > END; > END WaitPid; > > BEGIN > END SystemPosixWaitPidPause. > > questions: > > Aside: Where do I put "unsafe"? > > Relevant: Sysutils has to implement this itself, right? > Due to bootstrapping concerns. > > But we should still put /something/ in m3core, right? > > So then the big question, what to call it? > > Probably, the existing WaitProcess that does more than this WaitPid, should be scaled > back and only do what WaitPid does? > > I will likely commit exactly what is shown above. > The new interface and modules are in sysutils. The interface is private. > > The unfinished part is putting something in m3core, that sysutils could use, if > not for boostrapping issuers, that it could use in the future. > > I kind of thing something a bit more is needed. > You know, the Win32 implementation that asserts(false) is suspicious. > At the very least, it should be removed. > > I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. > Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 > interfaces? > > I also suspect part of the problem is that sysutils implements to a large extent > the same abstractions as m3core and/or libm3, but with variations, causing it to be unable > to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions > maybe are not general/featureful enough. > > (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) > > > ?? > > - Jay From mika at async.caltech.edu Wed Dec 31 08:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 30 Dec 2008 23:22:41 -0800 Subject: [M3devel] variations of waitpid..? In-Reply-To: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." Message-ID: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> I don't think it should be possible to stop other threads if the runtime could even conceivably support an environment where that's not necessary. You're introducing dependencies that will lead to (undetectable) deadlocks. People who declare their own <*EXTERNAL*> things are, as usual, on their own. Mika Jay writes: > >Hm. How about: > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? > >? >true for pthreads/win32/cygwin >false for userthreads > >duplicated privately in sysutils to avoid depending on newer m3core > like I did for the portable C-wrapped waitpid > >? > >That way..the other code can do whatever it already is doing, >no need to come up with abstractions for looping with nohang, >vs. that plus the status loophole. > >The older fix I put it can be shrunk. > >The name of this is debatable, but I think the minimal level of abstraction is > goodness. > >The name is actually perhaps too..complicated..like, what does it mean? >Who uses it? What do they do with it? > >It means that waiting on a process to end allows threads in the waiting proces >s to run. >Perhaps it is too low level. > > > - Jay > > > >---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Date: Wed, 31 Dec 2008 05:33:22 +0000 >> Subject: [M3devel] variations of waitpid..? >> >> >> variations of waitpid.. >> >> Hey, we already have, that I put in on my initiative, months ago: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix. >m3 calls Pause(). *) >> VAR >> result: int; >> statusM3: Uwaitpid.waitpid_status_t; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); >> IF result # 0 THEN EXIT END; >> Pause(Delay); >> END; >> 0 *> >> RETURN statusM3.w_Loophole; >> END WaitProcess; >> >> pthreads and cygwin: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 >> and >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) >> VAR >> statusM3: Uwaitpid.waitpid_status_t; >> BEGIN >> LOOP >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO >> IF r> 0 THEN >> RETURN statusM3.w_Loophole; >> END; >> < 0*> >> END; >> >> END; >> END WaitProcess; >> >> win32: >> PROCEDURE WaitProcess (pid: int): int = >> BEGIN >> >> RETURN 0; >> END WaitProcess; >> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po >int. >> >> However, now, to fix sysutils, we need something LIKE: >> >> UNSAFE INTERFACE SystemPosixWaitPid; >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; >> >> BEGIN >> END SystemPosixWaitPid. >> >> pthreads: >> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> BEGIN >> RETURN waitpid(pid, status, 0); >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidEfficient. >> >> user threads: >> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> VAR result: int; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); >> IF result # 0 THEN >> EXIT >> END; >> Thread.Pause(Delay) >> END; >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidPause. >> >> questions: >> >> Aside: Where do I put "unsafe"? >> >> Relevant: Sysutils has to implement this itself, right? >> Due to bootstrapping concerns. >> >> But we should still put /something/ in m3core, right? >> >> So then the big question, what to call it? >> >> Probably, the existing WaitProcess that does more than this WaitPid, should >be scaled >> back and only do what WaitPid does? >> >> I will likely commit exactly what is shown above. >> The new interface and modules are in sysutils. The interface is private. >> >> The unfinished part is putting something in m3core, that sysutils could use, > if >> not for boostrapping issuers, that it could use in the future. >> >> I kind of thing something a bit more is needed. >> You know, the Win32 implementation that asserts(false) is suspicious. >> At the very least, it should be removed. >> >> I suspect I am missing something though, as suggested by Win32 having an ass >ert(false) implementation. >> Either that needs to be removed, or somehow the code can be unified, using j >ust the existing m3core/libm3 >> interfaces? >> >> I also suspect part of the problem is that sysutils implements to a large ex >tent >> the same abstractions as m3core and/or libm3, but with variations, causing i >t to be unable >> to build on what is there, but having to reinvent..which suggests the m3core >/libm3 abstractions >> maybe are not general/featureful enough. >> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth >reads.) >> >> >> ?? >> >> - Jay From mika at async.caltech.edu Wed Dec 31 09:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 31 Dec 2008 00:22:41 -0800 Subject: [M3devel] TEXT & etc. Message-ID: <200812310822.mBV8Mf2j027994@camembert.async.caltech.edu> Hello everyone, I mentioned a week or two ago that I had run into horrific performance issues with CM3's TEXTs. It wasn't my intention to start a debate about Unicode and Mahjong characters, just to point out that there's a serious performance problem with CM3 in this area. I run into performance problems with CM3 from time to time because I am trying to maintain a fairly large amount of software written in Modula-3 and it appears to me that the SRC/PM3 compilers often pay better attention to performance issues than the newer CM3 compiler. Generally speaking I think Modula-3 needs to avoid getting into the Java situation: Java is squeezed between "fast enough" Python and "much more efficient" C++... Modula-3 I see as the only hope for getting the performance of C++ with the safety of Java. The problems I have seen with CM3 are in the following areas: 1. Thread mutex acquisition involving kernel calls (pthreads implementation) 2. ISTYPE and TYPECASE (appear much faster on PM3, don't know why, could just be a profiling issue). In any case TYPECASE is very slow because of having to search the type tree. This could be improved but I can't work on it because I always run into some kind of fatal problem trying to bootstrap the compiler. 3. The TEXT issue. I mentioned performance problems before. It turns out that it was showing up in another area. I upgraded my PowerBook a while back (to OS X 10.4) and of course everything stopped working, including at least half my Modula-3 programs. I thought it was a versioning problem with C libraries I was importing, but it turned out that in at least one case I was running out of stack space because of the calls to get_char on concatenated TEXTs. (Not very long ones, at that.) To get to the point. What follows is my new version of TextCat.m3, which I believe works, but I can't really say because I'm perennially incapable of bootstrapping CM3. The basic idea is that you can have *one* TextCat.T, at the "root" of your TEXT. It will flatten anything else. I don't see a simple way of modifying the CM3 TEXT implementation to do much better than this. Of course throwing out TextCat.T completely is an option. An advantage of what I've done is that I haven't changed any interfaces at all. So I don't get problems with version stamps, etc. Mika (* Copyright 1996-2000, Critical Mass, Inc. All rights reserved. *) (* See file COPYRIGHT-CMASS for details. *) UNSAFE MODULE TextCat EXPORTS RTHooks, TextCat; IMPORT TextClass, Text8, Text16; (* changed 12/31/2008: only return a T when neither of the arguments to Concat is already a T; otherwise flatten to a Text8 or Text16. Change affects only Concat and MultiCat *) REVEAL T = Public BRANDED "TextCat.T" OBJECT OVERRIDES get_info := MyGetInfo; get_char := MyGetChar; get_wide_char := MyGetWideChar; get_chars := MyGetChars; get_wide_chars := MyGetWideChars; END; PROCEDURE New (t, u: TEXT): TEXT = BEGIN RETURN Concat (t, u); END New; (* RTHooks.Concat -- called by the inline "&" operator *) PROCEDURE Concat (t, u: TEXT): TEXT = VAR ti, ui: TextClass.Info; BEGIN t.get_info (ti); IF (ti.length <= 0) THEN RETURN u; END; u.get_info (ui); IF (ui.length <= 0) THEN RETURN t; END; IF NOT ISTYPE(t,T) AND NOT ISTYPE(u,T) THEN RETURN NEW (T, a := t, b := u, a_len := ti.length, b_len := ui.length, a_or_b_wide := ti.wide OR ui.wide) ELSIF ti.wide OR ui.wide THEN WITH res = Text16.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_wide_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_wide_char(i) END; RETURN res END ELSE WITH res = Text8.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_char(i) END; RETURN res END END END Concat; PROCEDURE NewMulti (READONLY x: ARRAY OF TEXT): TEXT = BEGIN RETURN MultiCat (x); END NewMulti; (* RTHooks.MultiCat *) PROCEDURE MultiCat (READONLY x: ARRAY OF TEXT): TEXT = VAR wide := FALSE; length, start := 0; xi : TextClass.Info; BEGIN IF NUMBER (x) <= 0 THEN RETURN ""; END; IF NUMBER (x) = 1 THEN RETURN x[0]; END; IF NUMBER (x) = 2 THEN RETURN Concat(x[0],x[1]) END; FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); wide := wide OR xi.wide; length := length + xi.length END; IF wide THEN WITH res = Text16.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_wide_char(j) END; start := start + xi.length END; RETURN res END ELSE WITH res = Text8.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_char(j) END; start := start + xi.length END; RETURN res END END END MultiCat; PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; PROCEDURE MyGetChar (t: T; index: CARDINAL): CHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetChar; PROCEDURE MyGetWideChar (t: T; index: CARDINAL): WIDECHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_wide_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_wide_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetWideChar; PROCEDURE MyGetChars (t: T; VAR a: ARRAY OF CHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetChars; PROCEDURE MyGetWideChars (t: T; VAR a: ARRAY OF WIDECHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetWideChars; BEGIN END TextCat. From roland.illig at gmx.de Wed Dec 31 11:33:31 2008 From: roland.illig at gmx.de (Roland Illig) Date: Wed, 31 Dec 2008 11:33:31 +0100 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? Message-ID: <495B4A7B.6010505@gmx.de> Hi, when trying to build the simplest program, examples/hello, I get the following error messages: $ ~/usr.local/packages/cm3-5.4.0/bin/cm3 --- building in LINUXLIBC6 --- new source -> compiling Hello.m3 Hello.ms: Assembler messages: Hello.ms:21: Error: suffix or operands invalid for `push' Hello.ms:23: Error: suffix or operands invalid for `push' Hello.ms:27: Error: suffix or operands invalid for `pop' ... Is there some way of running cm3 on an x86_64 system? Roland From rcoleburn at scires.com Wed Dec 31 15:49:32 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 09:49:32 -0500 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <495B3F1E.1E75.00D7.1@scires.com> I agree with Tony about use of Modula-3 vs. C. Modula-3 is a systems programming language. We should be using it for systems programming, not C. My vote is to always use Modula-3, not C. Regards, Randy >>> Tony Hosking 12/30/2008 6:47 PM >>> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:28:37 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:28:37 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> Message-ID: <495B4846.1E75.00D7.1@scires.com> Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:46:27 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:46:27 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> References: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> Message-ID: <495B4C74.1E75.00D7.1@scires.com> Jay: The "cm3-41-hack.c" file is an old file that isn't used anymore. I seem to recall that the CMass folks created it for me back in the days of v4.1 to get around a bug. When I committed the cm3ide package a while back, I also committed the WindowsResoures package. I kept the cm3-41-hack.c file in there for historical purposes just in case someone ever has occasion to run cm3 v4.1. Indeed, anyone who purchased the CMass "Reactor" product (as I did) has v4.1 of cm3. You will note in the WindowsResources m3makefile that the cm3-41-hack.c file is commented out and there is a long comment proceeding it to explain its purpose. On a cm3 v4.1 system, you would either need to uncomment this line, or fix the underlying bug that prompted the need for it in the first place. Regards, Randy >>> Jay Krell 12/31/2008 11:03 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:03:11 Modified files: cm3/m3-sys/windowsResources/src/: cm3-41-hack.c Log message: microoptimize and reformat -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:58:44 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:58:44 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105714.24CF710D5DDE@birch.elegosoft.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> Message-ID: <495B4F55.1E75.00D7.1@scires.com> Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 18:59:03 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 12:59:03 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: <495B6B87.1E75.00D7.1@scires.com> Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy >>> 1/1/2009 5:25 AM >>> On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote: > > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation. It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine? > My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad. Except that the free download won't work on old versions of Windows. This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available. > I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents. > > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system. > > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own. It is not hard to write a linker in Modula-3. > From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment). What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice. > > Regards, > Randy -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:34:24 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:34:24 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:44:00 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:44:00 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4F55.1E75.00D7.1@scires.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy>>> Jay Krell 12/31/2008 11:57 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:57:14Added files:cm3/m3-sys/windowsResources/src/: m3overrides Log message:Add missing override to fix some builds.I'm highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:54:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:54:53 +0000 Subject: [M3devel] FW: [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: truncated, around the cmd/bat stuff From: jay.krell at cornell.eduTo: rcoleburn at scires.com; m3devel at elegosoft.comSubject: RE: [M3devel] [M3commit] CVS Update: cm3Date: Wed, 31 Dec 2008 18:34:24 +0000 > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view.They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT.They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns.Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided.They are truly identical in every respect in the front end and back end.They only vary in the config file.This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have.Therefore, BUILD_DIR is arbitrary.I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so..I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't.Show me where it does.The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific.I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe).I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH.True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python.cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB.That is also a competing workaround for paths with spaces.And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does).Personally I find cmd/bat among the worst programming languages ever and would rather not write it.jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows.That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it.Python is a very decent programming language that is very portable across "all" systems.Perl would be the next best choice, though..I just don't like much.sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win.I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself.It shouldn't be all that hard. Modula-3 still needs a C linker.And there is C code that I didn't put in -- hand.c and dtoa.c.If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning".You pretty much must chose one.The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:51:42 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:51:42 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: > It is not hard to write a linker in Modula-3. By all means then, do it.> What's hard is making it compatible with existing proprietary linkers > and loaders that are poorly documented and subject to change without > notice. Good point -- do you want to be able to debug with Visual Studio or cdb/ntsd/windbg? You are in for more work then. I don't think it as ever-changing as claimed, but it isn't easy either. For Windows, you should be able to write the symbols to the .exe or a separate .dbg file, rather than deal with the truly undocumented and changing .pdb file format (there is a third party book that mostly documents it). Do the other systems change a lot? If you look across a wide range of time, well, then, yeah, you get to handle the various COFF to ELF transitions, and Apple's rewrite every few years (68K => XCOFF/CFM/PEF => MachO). They are also largely unavoidably system specific, so writing it just once won't suffice. You know..you have to learn about and handle object file formats, at least three of them, and executable file formats, again at least three -- ELF, (MS/PE)COFF, and Mach-O. And then for debugging, there is CodeView, stabs, DWARF, and maybe others. If you only want gdb, then you can just stick with stabs. (Other options seem to crash cm3cg -- like when I fiddle around with -g vs. -ggdb vs. -gstabs.) Or you can just debug with printf and have no symbols...ugh... - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:05:26 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:05:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B6B87.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> <495B6B87.1E75.00D7.1@scires.com> Message-ID: > All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary > distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I am curious if anyone wants it too, but it is not difficult. Just run the vcvars32, set USE_DELAYLOAD=0 for versions prior to roughly 6.0 or 5.0spsomething (Modula-3-ism, and if anyone really wanted, could just remove it and back it just work always), and run the existing automation. It should just work. You don't have to be running on NT4, there isn't the global /usr/include business as on Unix. The headers and libraries are part of tools and not part of the operating system. Even using the latest toolsets might just work. The compiler and linker continue to output very generic stuff. The main area that platform dependencies creep in is the C runtime. For example, IsDebuggerPresent() is not on NT3.51 or Win95, but is used in newer C runtime. Some of the Interlocked functions as well, but you can also just use intrinsics and inlines for them, and then worry about the procesor and not the operating system. e.g. some of the Interlocked intrinsics/inlines require a 486, and some require something even newer -- the InterlockedCompareExchange64 stuff, if we have any. Even so, that's probably on all processors under 10 years old. XP requires it I think, but NT4/Windows 2000/Win9x did not. I think 486 is required for the modern variants of InterlockedIncrement/Decrement -- which return the entire new value, and not merely <0, 0, >0. InterlockedSomething is actually a small issue. The current C runtimes use InterlockedSomething that aren't exported on older (like pre-Win2000) systems. InterlockedIncrement/Decrement..kinda scary..you know, the functions changed meaning, in a compatible way, just from a binary's imports, you can't tell if it requires the newer meaning or if the older meaning suffices. To be clear, the newer meaning satisfies the older contract, but the older meaning doesn't satisfy the newer contract. Anyway, the modern way is to inline/intrinsic them and require a 486. But Modula-3's use of the C runtime is very small. And I dare say..we should try to remove it entirely... - Jay Date: Wed, 31 Dec 2008 12:59:03 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy>>> 1/1/2009 5:25 AM >>>On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote:> > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation.It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine?> My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad.Except that the free download won't work on old versions of Windows.This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available.> I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents.> > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system.> > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own.It is not hard to write a linker in Modula-3.> From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment).What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice.> > Regards,> Randy-- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:14:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:14:02 +0000 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? In-Reply-To: <495B4A7B.6010505@gmx.de> References: <495B4A7B.6010505@gmx.de> Message-ID: Mostly yes. There is not always a full assortment of runtime libraries or development/static libs, like on a minimal Debian system. In your config file, make sure to put -m32 on the cm3cg invocation, and perhaps -32 or --32 on the as/gas invocation. They may or may not help. Or hurt. (They can hurt in that older builds don't understand them and error right way. They can help if you have biarch tools that default to 64bit. They make no different on modern tools that are only 32bit or biarch tools that default to 32bit). Or just try config files. Though I don't test this out much. I can look into it soon..late next week at the latest. > Hello.ms:21: Error: suffix or operands invalid for `push' Can you show us Hello.ms line 21, or the entire Hello.ms file? There is also a "native" AMD64_LINUX port that you might try. "biarchness" is nice and convenient, to kind of get two systems in one, but it can also be confusing and easily misused. OpenBSD is interesting here -- never any biarchness, simpler. - Jay> Date: Wed, 31 Dec 2008 11:33:31 +0100> From: roland.illig at gmx.de> To: m3devel at elegosoft.com> Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu?> > Hi,> > when trying to build the simplest program, examples/hello, I get the> following error messages:> > $ ~/usr.local/packages/cm3-5.4.0/bin/cm3> --- building in LINUXLIBC6 ---> > new source -> compiling Hello.m3> Hello.ms: Assembler messages:> Hello.ms:21: Error: suffix or operands invalid for `push'> Hello.ms:23: Error: suffix or operands invalid for `push'> Hello.ms:27: Error: suffix or operands invalid for `pop'> ...> > Is there some way of running cm3 on an x86_64 system?> > Roland> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:24:07 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:24:07 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> References: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> Message-ID: You mean, this function is easy to misuse? > People who declare their own <*EXTERNAL*> Like waitpid exposed from m3core? waitpid is already easy to misuse, on a userthread system, leading to possible (though I think rare) deadlock. It is easy to misuse on pthreads, lead "just" to bad performance, and in fact I believe cm3 is doing this, via sysutils. This at least guides you between two patterns of use, and fix the perf of cm3/sysutils. On a userthread system, waitpid(pid, flags = 0) waits for the child process, with all parent threads suspended. Generally I doubt the child depends on parent threads progressing, but, yeah, that could deadlock, like if a parent thread is waiting to a file or stdin of the child, or reading a child's stdout. The various uses do waitpid(pid, flags = nohang) and then sleep and try again. pthreads just uses waitpid(pid, flags = 0) and all threads keep running. Win32 just uses WaitForSingleObject(process handle, timeout=infinite), and again all threads keep running. Just another problem with user threads, where pthreads and Win32 are fine. > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? WaitPROCESS always yields. It is WaitPID that may or may not. So I renamed the function to DoesWaitPidYield() and moved it to SchedulerPosix.i3. ok? - Jay> To: jay.krell at cornell.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] variations of waitpid..? > Date: Tue, 30 Dec 2008 23:22:41 -0800> From: mika at async.caltech.edu> > > I don't think it should be possible to stop other threads if the> runtime could even conceivably support an environment where that's> not necessary. You're introducing dependencies that will lead to> (undetectable) deadlocks. People who declare their own <*EXTERNAL*>> things are, as usual, on their own.> > Mika> > Jay writes:> >> >Hm. How about:> > > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN?> > > >?> >true for pthreads/win32/cygwin> >false for userthreads> > > >duplicated privately in sysutils to avoid depending on newer m3core> > like I did for the portable C-wrapped waitpid> > > >?> > > >That way..the other code can do whatever it already is doing,> >no need to come up with abstractions for looping with nohang,> >vs. that plus the status loophole.> > > >The older fix I put it can be shrunk.> > > >The name of this is debatable, but I think the minimal level of abstraction is> > goodness.> > > >The name is actually perhaps too..complicated..like, what does it mean?> >Who uses it? What do they do with it?> > > >It means that waiting on a process to end allows threads in the waiting proces> >s to run.> >Perhaps it is too low level.> > > > > > - Jay> >> >> >> >----------------------------------------> >> From: jay.krell at cornell.edu> >> To: m3devel at elegosoft.com> >> Date: Wed, 31 Dec 2008 05:33:22 +0000> >> Subject: [M3devel] variations of waitpid..?> >>> >>> >> variations of waitpid..> >>> >> Hey, we already have, that I put in on my initiative, months ago:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.> >m3 calls Pause(). *)> >> VAR> >> result: int;> >> statusM3: Uwaitpid.waitpid_status_t;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG);> >> IF result # 0 THEN EXIT END;> >> Pause(Delay);> >> END;> >> 0 *>> >> RETURN statusM3.w_Loophole;> >> END WaitProcess;> >>> >> pthreads and cygwin:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3> >> and> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *)> >> VAR> >> statusM3: Uwaitpid.waitpid_status_t;> >> BEGIN> >> LOOP> >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO> >> IF r> 0 THEN> >> RETURN statusM3.w_Loophole;> >> END;> >> < 0*>> >> END;> >>> >> END;> >> END WaitProcess;> >>> >> win32:> >> PROCEDURE WaitProcess (pid: int): int => >> BEGIN> >>> >> RETURN 0;> >> END WaitProcess;> >>> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po> >int.> >>> >> However, now, to fix sysutils, we need something LIKE:> >>> >> UNSAFE INTERFACE SystemPosixWaitPid;> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int;> >>> >> BEGIN> >> END SystemPosixWaitPid.> >>> >> pthreads:> >>> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> BEGIN> >> RETURN waitpid(pid, status, 0);> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidEfficient.> >>> >> user threads:> >>> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> VAR result: int;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG);> >> IF result # 0 THEN> >> EXIT> >> END;> >> Thread.Pause(Delay)> >> END;> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidPause.> >>> >> questions:> >>> >> Aside: Where do I put "unsafe"?> >>> >> Relevant: Sysutils has to implement this itself, right?> >> Due to bootstrapping concerns.> >>> >> But we should still put /something/ in m3core, right?> >>> >> So then the big question, what to call it?> >>> >> Probably, the existing WaitProcess that does more than this WaitPid, should > >be scaled> >> back and only do what WaitPid does?> >>> >> I will likely commit exactly what is shown above.> >> The new interface and modules are in sysutils. The interface is private.> >>> >> The unfinished part is putting something in m3core, that sysutils could use,> > if> >> not for boostrapping issuers, that it could use in the future.> >>> >> I kind of thing something a bit more is needed.> >> You know, the Win32 implementation that asserts(false) is suspicious.> >> At the very least, it should be removed.> >>> >> I suspect I am missing something though, as suggested by Win32 having an ass> >ert(false) implementation.> >> Either that needs to be removed, or somehow the code can be unified, using j> >ust the existing m3core/libm3> >> interfaces?> >>> >> I also suspect part of the problem is that sysutils implements to a large ex> >tent> >> the same abstractions as m3core and/or libm3, but with variations, causing i> >t to be unable> >> to build on what is there, but having to reinvent..which suggests the m3core> >/libm3 abstractions> >> maybe are not general/featureful enough.> >>> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of> > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth> >reads.)> >>> >>> >> ??> >>> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:30:40 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:30:40 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: <495B901F.1E75.00D7.1@scires.com> Jay: First, I do want to say thanks for all you are doing for the cause of Modula-3. I don't think we say thanks enough for what others do for us. THANK YOU! I don't want to be perceived as "complaining", rather I am trying to voice my opinion in the hopes of influencing future direction. Of course, since I'm not doing the work, I can only make suggestions. In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. Now, I'm not sure how much effect this deoptimization has (maybe little), but it is clear that in this case your implementation choice has favored the non-native implementation over the native one. IMO, tradeoffs of this type are not good. If one is trying to convince someone to use Modula-3, why would you want to give them a "deoptimized" version just to make it easier to support a non-native environment---indeed, one that they may not even want/use? If we have to make a trade-off, I say favor the native implementation always over the non-native one. What I'm trying to say here is that my experience as a software, systems, and program engineer is that I've always been forced to justify the cost/benefit of development tools for any project. Many times I've had to go head to head with folks higher up in my own organization or in the customer's organization whose preconceived opinions were based on rumor and what they've heard rather than actual factual hands-on experience. I want to pick the best tool for the job instead of being forced to pick an inferior tool because someone higher in the food chain demanded it based on faulty preconceived opinion. I like Modula-3. I've found that I am more productive using it. But, if the compiler doesn't produce efficient code, I lose ground in arguing with the higher-ups. As for Python, I've never ventured to learn it, so for me, it is something of a mystery. But you miss the point. I'm not arguing the merits of Python, I'm just saying that anything Modula-3 requires on top of what is provided by the standard host platform represents a potential obstacle or barrier to ease of use/implementation. It also sends the message that somehow Modula-3 is not complete on its own, we have get Python just to install and oh yes we need a C compiler and a linker and a ...? IMO, ultimately we need a turn-key download and install routine for Modula-3 that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install, I can run it on Windows, but if you give me an install routine that requires I first install something else, e.g. Python, then that becomes a barrier to the folks who don't know Python or already have it installed. Am I alone in this line of thought? If so, I'll just be quiet. BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. My problem is that right now, it is hard to understand all the dependencies for proper building of cm3 from scratch. Indeed, I've contributed some CMD/BAT install stuff in the past, but it is now out of date. Perhaps if there is a way we could better document the proper build order and dependencies, it would be easier for folks in the community to help in this area. I certainly would volunteer to do the CMD files as long as I had enough information to do it right. At one point, I thought we had a file that showed the package build order and dependencies. Not sure if the format of this file is well-documented or if the file is kept up-to-date. Regards, Randy >>> Jay 12/31/2008 1:34 PM >>> > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:39:54 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:39:54 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: <495B9248.1E75.00D7.1@scires.com> Jay: I was just trying to explain a bit about the history of and reason for the windowsResources package. As for m3overrides, I rarely use it anymore. At one point in time, we had a large project going on and the engineers used m3overrides for local testing before shipping to the main repository. That way, we avoided committing something that was obviously broken and causing other engineers to have downtime. I also seem to recall devising a neat way to use m3overrides to build against stubs and to create test editions of a module. Alas, it's been a long time since I've managed a project with many Modula-3 engineers working concurrently. Regards, Randy >>> Jay 12/31/2008 1:44 PM >>> "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 21:55:13 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 20:55:13 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B901F.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <495B901F.1E75.00D7.1@scires.com> Message-ID: > In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. It is such a slight deoptimization, I think it is ok. Are you comfortable with only slightly optimizing never inlining integrated backend? Not optimal. But it sure does build faster than gcc. Great for productivity. And its "always only slightly optimizing" mode generates much better code than gcc's default "no optimization at all" mode. But gcc with optimizing should be able to beat it handily -- e.g. inline. Do you use gcc's optimizer or not? I usually don't. I want to be able to debug line by line with a close correlation of binary to source. I have plenty of experience debugging optimized code, and it can be an interesting fun educational challenge, but unless I'm really bored, I'd rather not do it. If you don't use the optimizer, then again, not optimal. I wouldn't mind /some/ optimization, esp. some inlining and constant propagation, but I dislike it when the debugger jumps around with no clue as to what "line" of code I am on (because "lines" of code often don't exist in optimized code, a line can go away, or get split up, or become shared with other "lines", etc.). Are you aware of the cost of PushEFrame/try/raise/finally in Modula-3? Nobody seems to care much. Granted, SOLgnu/sun does it better, so people care.Almost anything is more efficient. Windows/x86 C/C++ does something similar but way more optimized. All other C/C++ systems use "stack walking" and incur little cost from the "try" alone. Though raise is often more expensive. Ok, granted, my deoptimization is to make PushEFrame even less efficient. It is already space (stack) and CPU inefficient. I made it more space (stack) inefficient. Could contribute to stack exhaustion. > IMO, ultimately we need a turn-key download and install routine for Modula-3 > that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install Can you try the uploaded archives I produced? Are they really too difficult to use?You extract them, optionally rename the root directory, run vcvars, add cm3 to %PATH%, and are done. Arguably much better than the older interactive cminstall. Not sure about the non-interactive variant. I understand your point. No Python is needed to download and run. No Python is really needed to build the system, but cm3 itself only builds a package at a time. The sh and Python automation is around cd'ing around and building, in the right order, with some filtering, and my Python adds ability to pick target platform on the command line, without messing around every time with config files and %PATH% and such. It is nice and convenient, it adds features that cm3 doesn't provide, but you don't have to have it. > BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. Your cmd/bat are actually still probably correct or very nearly so. The dependency orders rarely/never change. The main change is that sysutils was added. And perhaps that the dependency order became "more important" but not actually "different". That is, probably for a long time, an old cm3 could compile a current m3core/libm3. Now you need a cm3 with LONGINT support to build them (which itself is now a few years old). So you must first build the compiler with the old compiler, THEN build the current runtime. The "upgrade" scripts handle this. Again you can just cd around in the right order, with some repetition (you build the compiler first against the old runtime, then build the current runtime, then rebuild the compiler against the new runtime; arguably not needed but very much what you want to do). [redundancy here due to bad editing, sorry] Also, if you are bootstrapping from an older build, then you use their m3core and libm3, and then build the compiler up in dependency order starting with sysutils. If you are building from scratch with a current compiler, then you can start with m3core and go up from there. (Or import-libs on Windows, if needed, but this really just a workaround for old distributions, like 4.1, like before Tony's change to remove the "gcdefs" stuff.) > At one point, I thought we had a file that showed the package build order and dependencies. I was a big proponent of this, and it now exists and is in use..but I don't use it usually. :) It is the scripts/pkginfo.txt file. All the sh scripts use it I believe. Therefore the tinderbox uses it I believe. My cmd that I don't use (scripts/win) I think use it. My python code does not use it. Oops. Well, I think I did implement it at least, so I didn't push someone else to and then not use it.. :) (Pick your evil -- do something yourself, and then not use it, or push someone else to, and then not use it). Eh, I should just use it from the Python, not a big deal. Not only the build order, but the file also aids in package set selection. It doesn't do the filtering though, oh well. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Dec 13 19:03:08 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 13 Dec 2008 18:03:08 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a);END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:08:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:08:58 +0000 Subject: [M3devel] code duplication.. Message-ID: Anyone want to combine these identical files into one shared one somewhere? m3-mail/postcard/src/OSUtils.m3m3-mail/webcard/src/OSUtils.m3 - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:53:36 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:53:36 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 14 07:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 14 Dec 2008 17:23:06 +1100 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: > I tried getting C to error this way, and failed. > It looks like C uses COMPONENT_REF here, which will send it down > different paths I didn't follow. > COMPONENT_REF makes sense for it and only somewhat sense for us. > That would involve declaring all of a module's globals a struct, > which I don't believe we currently do but might be good. I think > there's a basic disconnect in that in C, bitfields are always > fields, never freestanding. Modula-3 rather violates that. Seems a > little dangerous therefore to be using them this way? > > In terms of the inefficient option in parse.c, that would maybe go > away, as all the offseting would be handled by the next level down, > like, implementation of fields? > Well..that is assuming the only use offset != 0 in m3_load is for > accessing module globals. I don't know if that is true. > > Maybe another option, should be easy to implement but might compile > slowly, is to cons up a field decl right there in m3_load/store? > That would lead to a lot of extra nodes -- per access instead of > declared once. I can try that out. > > - Jay > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sat, 13 Dec 2008 18:03:08 +0000 > > I've just spent a few hours debugging this. > > The problem is strongly related to the declaration that the global > segment is of size BIGGEST_ALIGNMENT. > gcc decides they can fit in a register..so then it doesn't bother > adding the offset to the address, something like that. > > When things work, it goes here: > op0 = adjust_address (op0, mode1, offset); > in expr.c. When things don't work, that line is skipped and then > shortly below that: > > if (mode1 != mode) > return convert_to_mode (tmode, op0, unsignedp); > return op0; << > > On systems where BIGGEST_ALIGNMENT fits in a register, e.g. > apparently MIPS, counter-e.g. apparently many others, but not > clearly all others. This determines whether the mode of the segment > is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger > than any machine type?). BLKmode works. It works on 32bit MIPS > because between mode1 and mode, one of them is SImode, one is > DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since > biggest alignment is set very large -- 128. I don't know why it > works for SPARC. > > PERHAPS I can put together a C or treelang example and get more (gcc > developers) eyes on it, but I doubt it. > > The following hack works around it: > > C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c > Index: parse.c > =================================================================== > RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v > retrieving revision 1.77 > diff -r1.77 parse.c > 2865c2865 > < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, > BIGGEST_ALIGNMENT); > --- > > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, > BIGGEST_ALIGNMENT); > > The repro is very easy. > Just add this to m3core: > > MODULE Main2 EXPORTS Main; > IMPORT RTIO; > VAR a := "a"; > BEGIN > RTIO.PutText(a); > END Main2. > > The load of a will be from MM_Main2 offset 0, instead of 104 or > thereabouts. > > I'll commit this shortly. > > - Jay > > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > Date: Thu, 13 Nov 2008 00:04:32 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > Jay, do you have access to an AMD64_DARWIN box? > > No, sorry, I don't yet and not sure when I will. > Unless it is easy enough to run on non-Apple hardware. > I'll have to debug it (m3cg) then. > > - Jay > > > > > > > > CC: m3devel at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Wed, 12 Nov 2008 20:46:53 +0000 > > Jay, do you have access to an AMD64_DARWIN box? I did get things > working on that 64-bit platform. Or is it an endianness issue you > are dealing with. In any case, I will not have a lot of bandwidth > for doing much in the way of tackling this unless it becomes > critical path for me soon. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 12 Nov 2008, at 07:57, Jay wrote: > > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:28:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:28:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct for size=align=maxalign, and then we access bit fields an offset from it. For if max is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:40:05 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:40:05 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: (truncated and reedited) Well, they are, definitely, but not with any fields, I think.Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it.If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang".Hopefully, going from just its name, it lets you build the exact trees that parse.c builds.As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supportedin current gcc without small patches (which cm3 applies).And then ask the gcc folks (cc you/all?)I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 10:20:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 09:20:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Treelang doesn't look hopeful. It is more like a "real language", though incomplete, and less about "direct tree building". I was hoping for something Lisp-like that mapped directly to the internal data structures. For now, I'm tired/lame/impatient, and will just stick with the "* 2" patch. Or maybe when we call relayout_decl, we can have a proper size. Maybe loads of module globals should go through some more specific than m3_load, and we can track the maximum offset. I'll try to write something reasonable to the gcc folks.. I could possibly write my own "treelang", but it doesn't seem worth the time currently, too much else to do. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: FW: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 14 Dec 2008 08:40:05 +0000 (truncated and reedited) Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it. If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 14 Dec 2008 17:23:06 +1100 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c Index: parse.c =================================================================== RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v retrieving revision 1.77 diff -r1.77 parse.c 2865c2865 < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT); --- > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main; IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a); END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu Date: Thu, 13 Nov 2008 00:04:32 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.com From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Wed, 12 Nov 2008 20:46:53 +0000 Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 10:01:58 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0 This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sat, 8 Nov 2008 22:55:54 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From jay.krell at cornell.edu Mon Dec 15 17:37:33 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 15 Dec 2008 16:37:33 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know. Not quite eveything is checked in and I didn't build a distribution yet. Within a few days, hopefully. I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 12:32:18 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 11:32:18 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass. Two are overly 32 bit specific. Two hang. Others fail that look familiar and/or floating point related, so probably fail on all. I didn't look too closely. The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap? I can try going back to that before debugging the hangs. In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited. Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 17:22:52 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 16:22:52 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I fixed the hang and updated the two archives. I had time_t wrong in some places. Are people on this list using any of: Linux/AMD64 FreeBSD/AMD64 NetBSD/AMD64 OpenBSD/AMD64 Solaris/AMD64 or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? or for that matter...? (I use Windows/AMD64 but am holding off on anything there.) - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Tue, 16 Dec 2008 11:32:18 +0000 please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass.Two are overly 32 bit specific.Two hang.Others fail that look familiar and/or floating point related, so probably fail on all.I didn't look too closely.The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs.In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited.Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Dec 16 20:33:27 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:33:27 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081215173318.0398710D5C74@birch.elegosoft.com> References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: This new file FilePosixC.c does not build on my SOLgnu Tinderbox setup, so the regressions failed last night. Jay, this is precisely the reason that we don't want to have C-coded files in the Modula-3 code-base -- C might proclaim its portability but that portability is a lie. Why do we need this when the old pure Modula-3 setup worked just fine! I know it meant some amount of duplication, but that duplication was needed because of differences in API specs on different platforms. I am not averse to having *portable* C code, but in this case I think it is too low-level to be properly portable. Perhaps better to keep it in Modula-3. On 15 Dec 2008, at 18:33, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/15 18:33:17 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > Log message: > Remove the need to untangle the #ifdefs around struct flock. From hosking at cs.purdue.edu Tue Dec 16 20:39:17 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:39:17 +1100 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: On 16 Dec 2008, at 22:32, Jay wrote: > please try out: > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > and/or > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > > Regression tests mostly pass. > Two are overly 32 bit specific. > Two hang. > Others fail that look familiar and/or floating point related, so > probably fail on all. > I didn't look too closely. > The hangs need investigation. > > Tony, should mmap work now, for arbitrarily distributed mmap? > I can try going back to that before debugging the hangs. Probably not. We still need a byte-sized entry per heap page that is maintained in the old array. I am working on developing a sparser representation for the 2 bits of information we still need to be able to extract given an arbitrary ambiguous root (i.e., something that looks like a pointer into the heap from the thread stacks). > > In one case I know the test didn't hang, but the test infra > structure hangs waiting for it. > > All relevant changes are commited. > Sorry this has taken so long.. > > - Jay > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Subject: RE: [M3devel] AMD64_LINUX (again) > Date: Mon, 15 Dec 2008 16:37:33 +0000 > > I never debugged the formsedit crash, but it seems to be gone now. > Possibly courtesy of pruning down the cloned headers, I don't know. > Not quite eveything is checked in and I didn't build a distribution > yet. > Within a few days, hopefully. > I need to try the regression tests too. > > - Jay > > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Mon, 17 Nov 2008 08:39:48 +0000 > Subject: [M3devel] AMD64_LINUX (again) > > ok, with the sbrk workaround now: > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > You don't have to build anything. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > You probably want to run ./do-cm3-std.py buildship. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz > You get to build your own of almost everything (./install- > config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py > buildship && ./install-front.py && ./do-cm3-std.py buildship) > > > OR you can bootstrap from an existing other system, roughly: > cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/ > python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py > AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target > machine and extract it, cd into it, make, and then you have native > cm3 and proceed to previous.. > > > I'm able to run Juno, Cube, Calculator, Mentor. > > Hm. formsedit crashes. Darn. > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 1140881760 (LWP 13270)] > 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Cannot > access memory at address 0x144006bf8 > ) > at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > 325 VBT.PaintTint(v, r, v.shadow.bg); > (gdb) c > Continuing. > Cannot find user-level thread for LWP 13242: generic error > (gdb) bt > #0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Ca > nnot access memory at address 0x144006bf8 > ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > Cannot fetch general-purpose registers for thread 1140881760: > generic error > (gdb) > > > still might be worth other people trying out.. > > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 01:32:06 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 00:32:06 +0000 Subject: [M3devel] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: Tony, Well, it is a matter of degree, but I really dislike the duplication and I believe the portability is plenty attainable. It also didn't compile for me on Cygwin, I did a bit more research and fixed that. It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't rebuild there yet. L_SET is not portable. - Jay> From: hosking at cs.purdue.edu> To: jkrell at elego.de> Date: Wed, 17 Dec 2008 06:33:27 +1100> CC: m3devel at elegosoft.com; m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3> > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > setup, so the regressions failed last night.> > Jay, this is precisely the reason that we don't want to have C-coded > files in the Modula-3 code-base -- C might proclaim its portability > but that portability is a lie. Why do we need this when the old pure > Modula-3 setup worked just fine! I know it meant some amount of > duplication, but that duplication was needed because of differences in > API specs on different platforms. I am not averse to having > *portable* C code, but in this case I think it is too low-level to be > properly portable. Perhaps better to keep it in Modula-3.> > On 15 Dec 2008, at 18:33, Jay Krell wrote:> > > CVSROOT: /usr/cvs> > Changes by: jkrell at birch. 08/12/15 18:33:17> >> > Modified files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3> > m3makefile> > Added files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c> >> > Log message:> > Remove the need to untangle the #ifdefs around struct flock.> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 01:49:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 11:49:36 +1100 Subject: [M3devel] [M3commit] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: I'll wait for tomorrow's regressions to come through and see if it is fixed now for my SOLgnu Tinderbox setup. I see the temptation to eliminate duplication, and for these sorts of POSIX APIs I would expect portable C code to work pretty well. On 17 Dec 2008, at 11:32, Jay wrote: > Tony, Well, it is a matter of degree, but I really dislike the > duplication > and I believe the portability is plenty attainable. It also didn't > compile > for me on Cygwin, I did a bit more research and fixed that. > It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't > rebuild there yet. > L_SET is not portable. > > - Jay > > > From: hosking at cs.purdue.edu > > To: jkrell at elego.de > > Date: Wed, 17 Dec 2008 06:33:27 +1100 > > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > > setup, so the regressions failed last night. > > > > Jay, this is precisely the reason that we don't want to have C-coded > > files in the Modula-3 code-base -- C might proclaim its portability > > but that portability is a lie. Why do we need this when the old pure > > Modula-3 setup worked just fine! I know it meant some amount of > > duplication, but that duplication was needed because of > differences in > > API specs on different platforms. I am not averse to having > > *portable* C code, but in this case I think it is too low-level to > be > > properly portable. Perhaps better to keep it in Modula-3. > > > > On 15 Dec 2008, at 18:33, Jay Krell wrote: > > > > > CVSROOT: /usr/cvs > > > Changes by: jkrell at birch. 08/12/15 18:33:17 > > > > > > Modified files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > > > m3makefile > > > Added files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > > > > > Log message: > > > Remove the need to untangle the #ifdefs around struct flock. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 02:00:35 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 01:00:35 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I asked mainly in case it was related to the hang, but it wasn't. Thanks, - Jay CC: m3devel at elegosoft.comFrom: hosking@ Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs. Probably not. ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 16 18:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 16 Dec 2008 11:26:05 -0600 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: <4947E4AD.8080705@wichita.edu> I am using Linux/AMD64. Jay wrote: > I fixed the hang and updated the two archives. > I had time_t wrong in some places. > > Are people on this list using any of: > Linux/AMD64 > FreeBSD/AMD64 > NetBSD/AMD64 > OpenBSD/AMD64 > Solaris/AMD64 > or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? > or for that matter...? > > (I use Windows/AMD64 but am holding off on anything there.) > > - Jay > From rcoleburn at scires.com Wed Dec 17 16:12:09 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 10:12:09 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> Message-ID: <4948D035.1E75.00D7.1@scires.com> Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 17:29:54 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 16:29:54 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4948D035.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:10:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:10:05 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:23:06 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: > Eh, not a big deal, you can cd to it and it just fine. > > I removed it from my distribution building and regular building. > It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering > wasn't working. > I "merely" need to fix one or the other or just comment it out when > building Cygwin, which is rare. > > See...I figured..that the Cygwin serial package should still use the > Win32 source. > However, that has a dependency on FileWin32.m3, which is only built > in "full Win32" platforms. > If you do try to compile FileWin32.m3, it errors, because it exports > File, and I guess maybe FileWin32. > Possibly I just need to have Quake do a bit of munging such that the > first line of FileWin32.m3 is altered. > The NT386 version would be: > MODULE FileWin32 EXPORTS File. > > The NT386GNU version would be: > MODULE FileWin32; > > something like that. > > Or push the contents into another module, and then have a thin NT386 > module export it as File. > > Anyway, it's not a huge deal. > The main downfall is I "temporarily" no longer build it, on any > platform. > > Fixing the filtering is a better "temporary" solution, since that'd > restore me to building it for all but NT386GNU. > > Another avenue is to see if the Posix version works on Cygwin. > I think the Cygwin serial users however are a "multiple small > intersection" -- people who use Modula-3 (small) intersected with > people who use Cygwin (medium) (probably at zero here) intersected > with people who use serial (very small) => very very very small, so > that's not worth getting it to work. > > - Jay > > > Date: Wed, 17 Dec 2008 10:12:09 -0500 > From: rcoleburn at scires.com > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > > Jay: > > Why are we skipping serial package? I use serial I/O on Windows. > If you need my sources, let me know and I'll supply. > > Regards, > Randy > > >>> Jay Krell 12/17/2008 3:47 PM >>> > CVSROOT:/usr/cvs > Changes by:jkrell at birch.08/12/17 15:47:34 > > Modified files: > cm3/scripts/python/: pylib.py make-dist.py > > Log message: > fix line endings and 'temporarily' skip serial package -- there's a > problem here on I386_CYGWIN I forgot about where essentially you > want to provide both FilePosix and FileWin32; FilePosix will export > File, FileWin32 will export FileWin32, so that portable Modula-3 can > use File/FilePosix and Win32 specific serial code can use FileWin32; > maybe some day we should try out this serial stuff, and /maybe/ the > Posix paths work on Cygwin? For now, just skip it. And fix line > endings. And try out the alternate 'clean' path in make-dist > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 22:36:36 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 21:36:36 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: I have a SOLsun/SOLgnu system at home. Wait a few more hours please? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:05:30 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:05:30 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: From: TonyTo: jay I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? It's more complicated than that. It isn't just one bit. There's a few factors, that people care more and less about. GUI library -- Win32 or X thread library -- pthreads or Win32 (or user threads) file system path representation -- forward slashes or backward slashes This seems to be what people care most about -- to see forward slashes. C compiler -- Visual C++ cl or Cygwin/GNU gcc linker -- Visual C++ link or Cygwin/GNU ld C runtime -- msvcr*.dll or cygwin1.dll integrated backend or not The integrated backend is way faster, but doesn't yet support 64bit LONGINT. which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/pdb Many of these factors are independent of each other, but not entirely. Compiler/linker directly lead to which debugger you can use, since they each use different formats. Well, you can use either debugger, but..without symbols..at least for some modules. The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBase, so you can't link to the Cygwin C runtime with theVisual C++ linker. This is probably fixable at least in one combination withan alias (ability to link Cygwin C runtime with Visual C++ linker; I don'tknow if Cygwin/GNU ld understands aliases). I have also found that Cygwin produces incorrectimport .libs, perhaps their linker somehow accepts them. Cygwin can program the Win32 GUI, but there is an issue with passing 8 byte structs by value to __stdcall functions, the function name isn't mangled correctly and linking fails. It is a bit of a combinatorial explosion.As the system stands, you can control each bit in the config file, andmultiple combinations might work, though I don't run any combinatorial tests. Three combinations are "identified" (named, spoken of) and have had some development testing.NT386, NT386MINGNU (which I think should be called I386_MINGWIN), and NT386GNU (which I think should be called I386_CYGWIN). Furthermore, you can imagine I386_INTERIX and I386_UWIN (I forget if that's the name).Interix and UWIN being two other very viable Posix layers on Windows.Interix has been renamed a few times. It is called "SFU" -- services for Unix, and I don't know,like SUA -- services for unix applications, and I think there's another name.It is a free download from Microsoft for x86 Windows < Vista, and is an optional component in Vista.It has its pluses and minuses..not now. Anyway, in conversation here, one of the main things that people DO NOT care aboutis which thread library is used. I believe it was asserted that PM3/NT386GNU did work, but that it might have used Win32 threads. And maybe never had any GUI? I tried getting pthreads to work, but gave up debugging it.Perhaps a fresh go at debugging it would pan out..but I'm really not too interested.. I also have no interest in user threads. I believe they are never warranted, excepton old systems that don't have kernel threads, like the hypothetical DJGPP port. I never brought up the issue of serial ports.I ASSUMED that almost nobody uses them, and anyone that does use them, doesn'tcare if they are implemented using Cygwin or Win32, as long as they work. I ASSUME there is no interop issue, like getting an int fd from the serial portand passing it to select() or close, or getting a Win32 HANDLE and passing itto WaitForSingleObject, but I could be wrong here, certainly. I didn't look into if Cygwin is complete enough to implement them.I consider serial ports to be kind of out on the fringe..that it wouldn't be surprising if Cygwin didn't implement them..but I didn't look. Heck, I tried building minicom on some non-Linux system(s) recently and it failed miserably. I don't know if there is even much Posix portability option for serial port programming. Sure..I debug over a serial port, and lately have machines I can't get videoout of (Itanium) or even seem to have no video hardware (Sun), so I use a serial consolefor initial install (recent achievement :) still haven't installed over network..), but that's still imho a big step away from writing code that uses the serial port.(And NetBSD doesn't support the local video on SGI O2, but I'm running OpenBSD there for now.) Anyway, there are a few ways to look at this issue.More and less trivial. A trivial view is: Get the filtering to work. Build serial on all platforms but Cygwin. Or see if the Posix code will work. I'd be surprised if the intersection of Cygwin x Modula-3 x serial port programmers is not 0, but could be. I actually don't think anyone is using Cygwin x Modula-3. Another view is: Get the Win32 serial code to work on Cygwin. The code is already there, and buildable, but building in a system that also builds FilePosix.m3 breaks Modula-3 rules, you end up with two implementations of the File interface. What you want, I think, is FilePosix.m3 to provide the File interface, and FileWin32.m3 to provide just its own specific FileWin32 interface. I think this is a simple transform of the code. Anyway, not a big deal...my tone here...is rushed, not rude, sorry.Maybe I'll get the truncation I deserve. :) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:07:33 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:07:33 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: <4949318C.1E75.00D7.1@scires.com> Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy >>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500 From: rcoleburn at scires.com To: jkrell at elego.de CC: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:24:38 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:24:38 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: <7D219C8E-022A-4196-8786-CA8F4278C010@cs.purdue.edu> Yeah, I would have thought of M3 on Cygwin as presenting a POSIX API (even if some system services like threads are implemented under the covers using Cygwin). To that extent, everything other than m3core, where threads live, should compile as a POSIX library. On 18 Dec 2008, at 09:07, Randy Coleburn wrote: > Jay, > > I agree with Tony. > > From my point of view, you are "breaking" the normal Windows install > in order to rectify a problem for Cygwin. I put "breaking" in > quotes because the install will appear to work, but the serial > package will be silently omitted. > > It may not be a "big deal" to you, but it presents another hurdle > for those who want to install on Windows, and this hurdle is > probably not documented yet. > > Regards, > Randy > > >>> Tony Hosking 12/17/2008 3:23 PM >>> > I would have thought of CygWin as a POSIX platform that just happens > to have Windows underneath. Did we go round this roundabout before? > > On 18 Dec 2008, at 03:29, Jay wrote: > >> Eh, not a big deal, you can cd to it and it just fine. >> >> I removed it from my distribution building and regular building. >> It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering >> wasn't working. >> I "merely" need to fix one or the other or just comment it out when >> building Cygwin, which is rare. >> >> See...I figured..that the Cygwin serial package should still use >> the Win32 source. >> However, that has a dependency on FileWin32.m3, which is only built >> in "full Win32" platforms. >> If you do try to compile FileWin32.m3, it errors, because it >> exports File, and I guess maybe FileWin32. >> Possibly I just need to have Quake do a bit of munging such that >> the first line of FileWin32.m3 is altered. >> The NT386 version would be: >> MODULE FileWin32 EXPORTS File. >> >> The NT386GNU version would be: >> MODULE FileWin32; >> >> something like that. >> >> Or push the contents into another module, and then have a thin >> NT386 module export it as File. >> >> Anyway, it's not a huge deal. >> The main downfall is I "temporarily" no longer build it, on any >> platform. >> >> Fixing the filtering is a better "temporary" solution, since that'd >> restore me to building it for all but NT386GNU. >> >> Another avenue is to see if the Posix version works on Cygwin. >> I think the Cygwin serial users however are a "multiple small >> intersection" -- people who use Modula-3 (small) intersected with >> people who use Cygwin (medium) (probably at zero here) intersected >> with people who use serial (very small) => very very very small, so >> that's not worth getting it to work. >> >> - Jay >> >> >> Date: Wed, 17 Dec 2008 10:12:09 -0500 >> From: rcoleburn at scires.com >> To: jkrell at elego.de >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] [M3commit] CVS Update: cm3 >> >> >> Jay: >> >> Why are we skipping serial package? I use serial I/O on Windows. >> If you need my sources, let me know and I'll supply. >> >> Regards, >> Randy >> >> >>> Jay Krell 12/17/2008 3:47 PM >>> >> CVSROOT:/usr/cvs >> Changes by:jkrell at birch.08/12/17 15:47:34 >> >> Modified files: >> cm3/scripts/python/: pylib.py make-dist.py >> >> Log message: >> fix line endings and 'temporarily' skip serial package -- there's a >> problem here on I386_CYGWIN I forgot about where essentially you >> want to provide both FilePosix and FileWin32; FilePosix will export >> File, FileWin32 will export FileWin32, so that portable Modula-3 >> can use File/FilePosix and Win32 specific serial code can use >> FileWin32; maybe some day we should try out this serial stuff, and / >> maybe/ the Posix paths work on Cygwin? For now, just skip it. And >> fix line endings. And try out the alternate 'clean' path in make-dist >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:27:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:27:36 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> Message-ID: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e- mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:30:07 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:30:07 -0500 Subject: [M3devel] serial2 package Message-ID: <494936D7.1E75.00D7.1@scires.com> Jay: I'm too busy right now to jump into the serial I/O stuff, so I am sending you a ZIP file containing my "serial2" package. This package is a replacement for the regular "serial" package. It has a WIN32 and a POSIX implementation, but the POSIX implementation was tailored for HP-UX and to my knowledge never tested on anything else. The WIN32 implementation works for me. It has some bug fixes and extras that the regular serial package does not and it has been tested on NT, 2000, and XP. You are welcome to use any part of this that is helpful. I know that emailing ZIP files is probably bad form these days, but you can scan it with your AV program. There are no binaries in here, just text files. Regards, Randy Coleburn -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: serial2.zip Type: application/x-zip-compressed Size: 24875 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Dec 17 23:32:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:32:53 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: I really don't think it's a big deal, to omit a minor package here or there. Imho people should be able to build whatever they need from source. Granted, I'm not where I want to be there yet, building the OS and such. I think most of the packages have no users at all. The web browsers in the tree for example seemed useless. I don't think Cygwin has any users either, so granted, this compromise was backwards. I could just comment it out for building Cygwin and not commit it. Anyway, I'll improve serial one way or the other. Like either just put it back, breaking Cygwin (if you use the .py files), or fix the filtering (could always filter in the m3makefile, easy). Randy, Tony's point I took to be different. His point is just let it build the Posix code. And indeed, I don't remember if I ever tried it. I just assumed serial programming to be a bit out on the fringe and not surprising if Cygwin doesn't support it, or even if it works on many Posix platforms at all. Serial is after all one of the packages with filtering "built in" just for it. I know minicom (just an example of a friendly serial app on GNU/Linux (e.g. Debian)) for example doesn't compile on some "Posix" platforms I have -- I forgot which I tried..could be any of AIX, Irix, OpenBSD... (was trying to bring up additional serial consoles besides a really noisy machine; the clear winner was Hyperterm on XP using a $20 USB to serial converter fyi...). - Jay Date: Wed, 17 Dec 2008 17:07:33 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy>>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building.It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working.I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source.However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms.If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32.Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered.The NT386 version would be:MODULE FileWin32 EXPORTS File. The NT386GNU version would be:MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal.The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin.I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:52:14 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:52:14 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some. Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable" You throw this word around with emphasis a lot. Portable Modula-3 code. The lie of portable C code. What do you mean? There are many ways to write portable and non-portable Modula-3 and C code. For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*. It's quite easy to do either way, in either language. I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile. They are both actually very portable, but less portable than calling libraries that wrap them. What I mean is, Posix open is implemented on tons of systems, but not all. Win32 CreateFile is also implemented on tons of systems, but not all. It depends partly how you quantify systems. Win32 Create 1) is implemented on over a billion actual machines. open is actually in msvcrt.dll also, but let's ignore it for this discussion. I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever. I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. (Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything. I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library. It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C. If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix. I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting. That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable. Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained. As well, there is OpenMP. Another angle actually you can play here is quality of compiler. I did some experimentation with a range of compilers I had..I guess I was using C++ at the time. Some older compilers really stunk. Like, with CFront, you couldn't have more than one return. Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation. Cheating I think. See also Python and Perl. Well, ok, Modula-3 wins here by virtue of having a good small language spec. You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Dec 18 00:13:12 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 23:13:12 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: ps: similar question: What does "low level" mean? hand.c and dtoa.c seem low level to me. I know, they are excused because they are rare and in m3core. FilePosixC.c does not. The stuff I did for stat, somewhat. The garbage collector does. But I can't define it, other than, uselessly "dealing with bits is low level", but int is_even(int a) { return !(a & 1); } doesn't seem very "low level", just slightly, and is very "portable". I was given a book about "complexity" as a gift. It makes a point that "complex" and "simple" are hard to define. Many things that seem the one, also seem the other. I have to say, I have whittled down unix/*.i3 a bunch, for all the "new" platforms, and I look at what is left, and I am inclined to keep "attacking" it, keep making it "smaller", like the bitflags in Ustat.i3. I am inclined to write something like this: Ustat.i3: TYPE StatFlags = RECORD X, R, W: INTEGER; (* maybe u_short *) END; PROCEDURE GetStatFlags(VAR StatFlags); UstatC.c typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; const static StatFlags_t StatsFlags = { X, R, W }; void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; Any module using any of the flags, would retrieve them in initialization. A bit inefficient. Portable. Cuts out many repeated lines. The C could would need a bit of #ifdefing, to handle platforms that don't have some of the flags, to make them 0. Again, very portable. Cuts out repetition. Is a little slower. I know it also might not work, like if there are switch statements or code needing actual constant constants. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] Fwd: Output from "cron" commandDate: Wed, 17 Dec 2008 22:52:14 +0000 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some.Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable"You throw this word around with emphasis a lot.Portable Modula-3 code.The lie of portable C code.What do you mean? There are many ways to write portable and non-portable Modula-3 and C code.For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*.It's quite easy to do either way, in either language.I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile.They are both actually very portable, but less portable than calling libraries that wrap them.What I mean is, Posix open is implemented on tons of systems, but not all.Win32 CreateFile is also implemented on tons of systems, but not all.It depends partly how you quantify systems.Win32 Create 1) is implemented on over a billion actual machines.open is actually in msvcrt.dll also, but let's ignore it for this discussion.I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever.I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad.(Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything.I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library.It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C.If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix.I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting.That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable.Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained.As well, there is OpenMP. Another angle actually you can play here is quality of compiler.I did some experimentation with a range of compilers I had..I guess I was using C++ at the time.Some older compilers really stunk. Like, with CFront, you couldn't have more than one return.Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation.Cheating I think. See also Python and Perl.Well, ok, Modula-3 wins here by virtue of having a good small language spec.You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Dec 18 00:38:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 10:38:06 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: <292A034D-ED30-407E-B773-F7DB513524EF@cs.purdue.edu> Think about the difference between safe and UNSAFE in Modula-3. On 18 Dec 2008, at 10:13, Jay wrote: > ps: similar question: > > What does "low level" mean? > > hand.c and dtoa.c seem low level to me. > I know, they are excused because they are rare and in m3core. > > FilePosixC.c does not. > The stuff I did for stat, somewhat. > > The garbage collector does. > But I can't define it, other than, uselessly "dealing with bits is > low level", but > int is_even(int a) { return !(a & 1); } doesn't seem very "low > level", just slightly, and is very "portable". > > I was given a book about "complexity" as a gift. > It makes a point that "complex" and "simple" are hard to define. > Many things that seem the one, also seem the other. > > I have to say, I have whittled down unix/*.i3 a bunch, for all the > "new" platforms, and I look at what is left, and I am inclined to > keep "attacking" it, keep making it "smaller", like the bitflags in > Ustat.i3. > > I am inclined to write something like this: > > Ustat.i3: > > TYPE StatFlags = RECORD > X, R, W: INTEGER; (* maybe u_short *) > END; > > PROCEDURE GetStatFlags(VAR StatFlags); > > UstatC.c > > typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; > const static StatFlags_t StatsFlags = { X, R, W }; > void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; > > Any module using any of the flags, would retrieve them in > initialization. > A bit inefficient. > Portable. > Cuts out many repeated lines. > > The C could would need a bit of #ifdefing, to handle platforms that > don't have some of the flags, to make them 0. > > Again, very portable. > Cuts out repetition. > Is a little slower. > I know it also might not work, like if there are switch statements > or code needing actual constant constants. > > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] Fwd: Output from "cron" command > Date: Wed, 17 Dec 2008 22:52:14 +0000 > > 1) I totally don't mind the emails. I can ignore them for a day or > so easily. :) > > > 2) C code in m3core or elsewhere. This is a philisophical question I > couldn't decide, interested in discussing some. > Originally I put "FilePosixC.c" in m3core, and called it from libm3. > But I think that's..not great. > > > 3) "portable" > You throw this word around with emphasis a lot. > Portable Modula-3 code. > The lie of portable C code. > What do you mean? > > > There are many ways to write portable and non-portable Modula-3 and > C code. > For example there were Modula-3 regression tests printing the size > of an integer. > Portable to lots of systems, but not all. > > I can write Modula-3 code or C code dependent or independent of > sizeof(int) or long or void*. > It's quite easy to do either way, in either language. > I don't think that proves anything. > > > Likewise, in both Modula-3 and C, I can easily directly call Posix > open, or Win32 CreateFile. > They are both actually very portable, but less portable than calling > libraries that wrap them. > What I mean is, Posix open is implemented on tons of systems, but > not all. > Win32 CreateFile is also implemented on tons of systems, but not all. > It depends partly how you quantify systems. > Win32 Create 1) is implemented on over a billion actual machines. > open is actually in msvcrt.dll also, but let's ignore it for this > discussion. > I've read that the "Unix workstation" market is only like in the > single digit millions, either sold per year or sold ever. > I realize those billion PCs that run Windows, can also run *BSD and > Linux, making this all gray. > > Now, another way to quantify this though, that I find "impressive", > is that Win32 CreateFile at least historically was implemented on > PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. > (Basically, NT has been ported among the most of any /commercial > closed source/ OS.) > > But again, I don't think this proves anything. > I can pretty equally easily write portable or non-portable C or > Modula, "at the library level". > > One angle you might argue, is that Modula-3 has a "built in" > portability layer/library. > It is, in a sense, easier to not call CreateFile or open in Modula-3 > than it is in C. > If that is what you mean, ok. > > HOWEVER, if you look at the Modula-3 system, the portability layer, > you know, it is forked in terms of Posix and Win32. If you limit > yourself to the Posix part of it, well, then, you can use Posix. > I realize Posix systems are not all equal, they are all supersets of > Posix, with some overlap in their supersetting. > That's why my checkin worked on some systems but not others. > > Also, with respect to I/O, if use just stdio -- very portable. > Threading -- finally, soon, multi threading will be very portable in > C++, at least to the implementations still being maintained. > As well, there is OpenMP. > > Another angle actually you can play here is quality of compiler. > I did some experimentation with a range of compilers I had..I guess > I was using C++ at the time. > Some older compilers really stunk. Like, with CFront, you couldn't > have more than one return. > Therefore, code like: > int even(int a) > { > if (a & 1) > return 0; > return 1; > } > > /could/ be deemed not portable. > > Modula-3 wins here by virtue of there being only one implementation. > Cheating I think. > > See also Python and Perl. > Well, ok, Modula-3 wins here by virtue of having a good small > language spec. > You know..Perl is speced as the language implemented by > perl.exe....so big and gnarly and impossible to predict..impossible > to write down...C++ is fairly well speced, but only through > mountains of paper and endless discussion and clarifications.... > > > - Jay > > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 09:27:36 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Fwd: Output from "cron" command > > > > No, broken briefly then fixed quickly is OK -- that's what > regressions are for. But on the other hand, testing before checkin > is probably nicer to the rest of the world. That way you don't get > whinging e-mails from me complaining that the Tinderbox regressions > broke. I suppose it depends which pain you prefer: my e-mails or > testing comprehensively. ;-) > > I would hope that these sorts of system-dependent changes are > minimized by keeping as much code in *portable* Modula-3 as > possible. To my mind, nothing outside of m3core should need to > escape to C. > > On 18 Dec 2008, at 09:24, Jay wrote: > > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 04:10:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:10:15 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." Message-ID: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- From mika at async.caltech.edu Thu Dec 18 04:33:46 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:33:46 -0800 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: Your message of "Wed, 17 Dec 2008 22:52:14 GMT." Message-ID: <200812180333.mBI3XktJ079016@camembert.async.caltech.edu> I don't know what kind of a programming Superman you are. I switched from C to Modula-3 in 1999 because I was tired of having a new crop of bugs every time my C programs touched a new OS. Or, on Linux, every time there was a new version of Linux out (i.e., every day, twice on Sundays?). Yes, they were in fact flaws in my code. I don't doubt that there could exist bug-free portable programs coded in C that do what my programs were supposed to do. But in my experience, very very few programmers know C well enough to write truly portable programs. I'm also not one of them even though I've written more C than I care to remember. In Modula-3, it just seems natural...in fact it takes effort to write a program that *isn't* portable, the opposite of C. In fact, when I was teaching CS2 at Caltech, our CS1-2-3 sequence was sometimes known as the "Useless Languages Sequence" (it was supposed to be intro to programming, program construction/basic algorithms and data structures, and software engineering). CS1 was MIT's 6.001 (in Scheme); CS2 I taught in Modula-3; and someone taught CS3 in Eiffel. Each of the years I taught CS2 I had some students babbling on about how they wanted to study the material in C or C++, because they knew it and thought the other students would be more employable (it would be good for them). Well these self-professed C experts as good as never could answer some basic questions about C that have some very odd answers... For instance, if the following is a complete C program... int main() { int a[10]; int *b; b=... } how many values can b (legally) have? I believe the answer is... 13? Don Knuth once sent me a check for $2.56 because I spotted some non-conforming C in Vol. 2... If you don't know all those rules and think about them CONSTANTLY you will one day run into a compiler that makes you a very unhappy programmer. Sigh. I remember debugging a program that was comparing function pointers for ordering once... on a Convex. Yes the program was incorrect---unlike any other types of pointers, function pointers can only be compared for equality---so you might say we got what we deserved? (Of course the bloody thing did precisely what the programmer wanted on SunOS/sparc, NetBSD/i386, OSF/1 on the alpha, NetBSD on the hp300, solaris/sparc and every other OS we ever tried it on; and it didn't crash immediately nor emit warnings on the Convex but crashed sometimes minutes (millions of instructions) later.) Well @#%@#%%@ like that just doesn't happen in Modula-3. Mika Jay writes: >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :) >=20 >=20 >2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some. >Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great. >=20 >=20 >3) "portable" >You throw this word around with emphasis a lot. >Portable Modula-3 code. >The lie of portable C code. >What do you mean? >=20 >=20 >There are many ways to write portable and non-portable Modula-3 and C code. >For example there were Modula-3 regression tests printing the size of an in= >teger. > Portable to lots of systems=2C but not all. >=20 >I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*. >It's quite easy to do either way=2C in either language. >I don't think that proves anything. >=20 >=20 >Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile. >They are both actually very portable=2C but less portable than calling libr= >aries that wrap them. >What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll. >Win32 CreateFile is also implemented on tons of systems=2C but not all. >It depends partly how you quantify systems. >Win32 Create 1) is implemented on over a billion actual machines. >open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion. >I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever. >I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray. >=20 >Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently x86=2C IA64=2C AMD64. Not bad. >(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.) >=20 >But again=2C I don't think this proves anything. >I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level". >=20 >One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library. >It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C. >If that is what you mean=2C ok. >=20 >HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix. >I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting. >That's why my checkin worked on some systems but not others. >=20 >Also=2C with respect to I/O=2C if use just stdio -- very portable. >Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained. >As well=2C there is OpenMP. >=20 >Another angle actually you can play here is quality of compiler. >I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time. >Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return. >Therefore=2C code like: > int even(int a) =20 > { if (a & 1) return 0=3B =20 > return 1=3B =20 > } >=20 >/could/ be deemed not portable. >=20 >Modula-3 wins here by virtue of there being only one implementation. >Cheating I think. >=20 >See also Python and Perl. >Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec. >You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications.... >=20 >=20 > - Jay > > > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output= > from "cron" command > > >No=2C broken briefly then fixed quickly is OK -- that's what regressions ar= >e for. But on the other hand=2C testing before checkin is probably nicer t= >o the rest of the world. That way you don't get whinging e-mails from me c= >omplaining that the Tinderbox regressions broke. I suppose it depends whic= >h pain you prefer: my e-mails or testing comprehensively. =3B-) > >I would hope that these sorts of system-dependent changes are minimized by = >keeping as much code in *portable* Modula-3 as possible. To my mind=2C not= >hing outside of m3core should need to escape to C. > >On 18 Dec 2008=2C at 09:24=2C Jay wrote: > >(critical typo -- you can provide a time for it to power back on.)I'm loath= >e to leave all my machines on and burn the electricity..I don't trust them = >to have good power management=2C lower power when idle.I should automate so= >mething here though=2C see if I can run Tinderbox=2C and see if I can get c= >ron to power on/off.Every time I have looked at the Tinderbox it seemed too= > difficult to run. On the other hand=2C I don't know if the status quo is s= >o bad.You tell me it's broken. I understand that is not ideal=2C and I shou= >ld fix it fairly asap=2C but is it terrible every so often?I do tend to at = >least build multiple platforms=2C even if not "all"..so not every checkin b= >reaks anything/everything. I figure LINUXLIBC6 is the most popular=2C and I= > can always test that on birch. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] = >Fwd: Output from "cron" commandDate: Thu=2C 18 Dec 2008 09:12:52 +1100 > > >My machine is in a machine room at Purdue=2C on all the time. > >On 18 Dec 2008=2C at 08:37=2C Jay wrote: > >I keep most of my machines powered off most of the time. Do you have automa= >tion that both runs a daily Tinderbox=2C and turns the machine on/off?My SG= >I machine at least has a nice feature where you can software power it off= >=2C and provide a time that it should power down. I haven't automated=2C bu= >t it seems ideal for daily Tinderboxes. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output fro= >m "cron" command > > >Jay=2C > >I am still getting a compile error on whatever changes you made... > >-- Tony > >Begin forwarded message: > > >From: Tony Hosking >Date: 18 December 2008 00:04:32 GMT+11:00 >To: hosking at cs.purdue.edu >Subject: Output from "cron" command > >Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.s= >hproduced the following output:TESTHOSTNAME=3DniagaraWS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=3D5.4.0INSTROOT_REL=3D/homes/h= >osking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm= >3-inst/niagara/prev-okINSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/l= >ast-okINSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE= >=3DPOSIXCM3_TARGET=3DSOLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-S= >OLgnu-5.4.0.tgzCM3CVSSERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegoso= >ft.com:/usr/cvsBINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.eleg= >osoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Bu= >ildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file = >/tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout=2C compile and tes= >t of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 200= >8.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return v= >alue: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN =3D 0--2008.12.17 = >06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile = >return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN =3D 1*** COM= >PILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cle= >aning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regress= >ion test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3te= >st log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hoskin= >g/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.= >stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-= >min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLg= >nu-*.htmlTESTHOSTNAME=3DniagaraWS=3D/homes/hosking/work/cm3-ws/niagara-2008= >-12-17-11-57-02LASTREL=3D5.4.0INSTROOT_REL=3D/homes/hosking/work/cm3-inst/n= >iagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok= >INSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=3D/= >homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=3DPOSIXCM3_TARGET=3DS= >OLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSS= >ERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvsBINDIST= >MIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=3D/homes/hosking/work/c= >m3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.ele= >gosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu = >SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-0= >65704-RGaGCn/log.txt---checkout=2C compile and test of cm3 ...2008.12.17 06= >:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/= >build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008= >.12.17 07:16:18]CHECKOUT_RETURN =3D 0--2008.12.17 07:16:21 -- compile in pr= >ogress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compi= >le 2008.12.17 08:01:31]COMPILE_RETURN =3D 02008.12.17 08:01:38 -- tests in = >progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-06= >5704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN =3D 02008.= >12.17 08:01:38 -- checkout=2C compile and test run done.---removing build t= >ree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../hom= >es/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes= >/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hoski= >ng/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*= >.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snap= >shot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcle= >aning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone.= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :)
> =3B
> =3B
>2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some.
>Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great.
> =3B
> =3B
>3) "portable"
>You throw this word around with emphasis a lot.
>Portable Modula-3 code.
>The lie of portable C code.
>What do you mean?
> =3B
> =3B
>There are many ways to write portable and non-portable Modula-3 and C code.= >
>For example there were Modula-3 regression tests printing the size of an in= >teger.
> =3B Portable to lots of systems=2C but not all.
> =3B
>I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*.
>It's quite easy to do either way=2C in either language.
>I don't think that proves anything.
> =3B
> =3B
>Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile.
>They are both actually very portable=2C but less portable than calling libr= >aries that wrap them.
>What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll.
>Win32 CreateFile is also implemented on tons of systems=2C but not all.
>It depends partly how you quantify systems.
>Win32 Create 1) is implemented on over a billion actual machines.
>open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion.
>I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever.
>I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray.
> =3B
>Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently =3Bx86=2C IA64=2C AMD64. Not bad.
>(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.)
> =3B
>But again=2C I don't think this proves anything.
>I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level".
> =3B
>One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library.
>It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C.
>If that is what you mean=2C ok.
> =3B
>HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix.
>I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting.
>That's why my checkin worked on some systems but not others.
> =3B
>Also=2C with respect to I/O=2C if use just stdio -- very portable.
>Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained.
>As well=2C there is OpenMP.
> =3B
>Another angle actually you can play here is quality of compiler.
>I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time.
>Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return.
>Therefore=2C code like:
> =3B int even(int a) =3B
> =3B { =3B
 =3B  =3B if (a &=3B 1) =3B
 = >=3B  =3B =3B =3B return 0=3B =3B
> =3B  =3B return 1=3B =3B
> =3B }
> =3B
>/could/ be deemed not portable.
> =3B
>Modula-3 wins here by virtue of there being only one implementation.
>Cheating I think.
> =3B
>See also Python and Perl.
>Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec.
>You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications....
> =3B
> =3B
> =3B- Jay


> >
>
>From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Thu=2C 18= > Dec 2008 09:27:36 +1100
CC: m3devel at elegosoft.com
Subject: Re: [M3de= >vel] Fwd: Output from "cron" command


>
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >
No=2C broken briefly then fixed quickly is OK -- that's what regressio= >ns are for.  =3BBut on the other hand=2C testing before checkin is prob= >ably nicer to the rest of the world.  =3BThat way you don't get whingin= >g e-mails from me complaining that the Tinderbox regressions broke.  = >=3BI suppose it depends which pain you prefer: my e-mails or testing compre= >hensively.  =3B=3B-)
>

>
I would hope that these sorts of system-dependent changes are minimize= >d by keeping as much code in *portable* Modula-3 as possible.  =3BTo my= > mind=2C nothing outside of m3core should need to escape to C.
=

>
>
On 18 Dec 2008=2C at 09:24=2C Jay wrote:

erchange-newline> >
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>(critical typo -- you can provide a time for it to power back on.)
I'm = >loathe to leave all my machines on and burn the electricity..
I don't tr= >ust them to have good power management=2C lower power when idle.
I shoul= >d automate something here though=2C see if I can run Tinderbox=2C and see i= >f I can get cron to power on/off.
Every time I have looked at the Tinder= >box it seemed too difficult to run.
 =3B
On the other hand=2C I d= >on't know if the status quo is so bad.
You tell me it's broken. I unders= >tand that is not ideal=2C and =3BI should fix it fairly asap=2C but is = >it terrible every so often?
I do tend to at least build multiple platfor= >ms=2C even if not "all"..so not every checkin breaks anything/everything.R> =3B
I figure LINUXLIBC6 is the most popular=2C and I can always t= >est that on birch.
 =3B
 =3B- Jay


>
>
From: =3Bailto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:EC_Apple-converted-space> =3B.edu">jay.krell at cornell.edu
Subject: Re: [M3devel] Fwd: Output from = >"cron" command
Date: Thu=2C 18 Dec 2008 09:12:52 +1100


>
T: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-= >INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLL= >APSE: separate"> >
tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica= >=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WH= >ITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"><= >SPAN class=3DEC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12p= >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate">-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb= >(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: n= >ormal=3B BORDER-COLLAPSE: separate">yle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B= > COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER= >-SPACING: normal=3B BORDER-COLLAPSE: separate">yle-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFO= >RM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: norma= >l=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >
My machine is in a machine room at Purdue=2C on all the time.
PAN>
> >
>
On 18 Dec 2008=2C at 08:37=2C Jay wrote:

interchange-newline> >
=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate"> >
na">I keep most of my machines powered off most of the time.
 =3B>Do you have automation that both runs a daily Tinderbox=2C and turns the m= >achine on/off?
My SGI machine at least has a nice feature where you can = >software power it off=2C and provide a time that it should power down. I ha= >ven't automated=2C but it seems ideal for daily Tinderboxes.
 =3B> =3B- Jay


>
>From: =3Bilto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:C_EC_Apple-converted-space> =3Bll.edu">jay.krell at cornell.edu
Date: Thu=2C 18 Dec 2008 07:10:05 +110= >0
CC: =3B"mailto:m3devel at elegosoft.com">m3devel at elegosoft.com
Subject: [M3dev= >el] Fwd: Output from "cron" command


>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate"> >
n style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: non= >e=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LE= >TTER-SPACING: normal=3B BORDER-COLLAPSE: separate">pple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-= >TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE= >: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">s=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helv= >etica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate">FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate">ACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0= >=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: nor= >mal=3B BORDER-COLLAPSE: separate">tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">ple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-T= >RANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE:= > normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helve= >tica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate"> >
Jay=2C
>

>
I am still getting a compile error on whatever changes you made...V> >

>
-- Tony
<= >/DIV>
>

>
Begin forwarded message:

ewline> >
>
>
= >ica color=3D#000000 size=3D3>From:ace> =3Belvetica size=3D3>Tony Hosking <=3Bu">hosking at cs.purdue.edu>=3B
>
= >ica color=3D#000000 size=3D3>Date:ace> =3Belvetica size=3D3>18 December 2008 00:04:32 GMT+11:00
>
= >ica color=3D#000000 size=3D3>To:e> =3Bvetica size=3D3>hosking at cs.purdue.= >edu
>
= >ica color=3D#000000 size=3D3>Subject:-space> =3B=3DHelvetica size=3D3>Output from "cron" command
>
MARGIN-RIGHT: 0px">
>
Your "cron" job on niagara.cs.purdue.edu
$HOME/cm3/scripts/regressi= >on/cron.sh

produced the following output:

TESTHOSTNAME=3Dniag= >ara
WS=3D/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04
LASTR= >EL=3D5.4.0
INSTROOT_REL=3D/homes/hosking/work/cm3-inst/niagara/rel-5.4.0= >
INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok
INSTROOT= >_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-ok
INSTROOT_CUR=3D/home= >s/hosking/work/cm3-inst/niagara/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET= >=3DSOLgnu
BINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tg= >z
CM3CVSSERVER=3Dbirch.elegosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com= >:/usr/cvs
BINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.= >elegosoft.com..
ssh birch.elegosoft.com ok
Building cm3.
Tinderbox= > Tree:  =3B =3B"cm3"
Buildname:  =3B =3B =3B =3B= > =3B =3B =3B"SOLgnu SunOS 5.10 niagara release-build"

cr= >eating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt

--->
checkout=2C compile and test of cm3 ...
2008.12.17 06:30:07 -- chec= >kout in progress.
[start checkout 2008.12.17 06:30:11]
cd /tmp/build-= >cm3-20081217-063006-fOaGeh/build
cvs return value: 0
[end checkout 20= 08.12.17 06:49:43]
CHECKOUT_RETURN =3D 0
--
2008.12.17 06:49:45 --= > compile in progress.
[start compile 2008.12.17 06:49:45]
compile ret= >urn value: 0
[end compile 2008.12.17 06:55:03]
COMPILE_RETURN =3D 1R>*** COMPILE FAILED
removing build tree /tmp/build-cm3-20081217-063006-= >fOaGeh ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/niag= >ara-*

cleaning regression test log files...
/homes/hosking/tmp/cm= >3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking/= >tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3te= >sts-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract= >

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-mi= >n-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-r= >eport-SOLgnu-*.html

TESTHOSTNAME=3Dniagara
WS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-57-02
LASTREL=3D5.4.0
INSTROOT_REL=3D= >/homes/hosking/work/cm3-inst/niagara/rel-5.4.0
INSTROOT_POK=3D/homes/hos= >king/work/cm3-inst/niagara/prev-ok
INSTROOT_LOK=3D/homes/hosking/work/cm= >3-inst/niagara/last-ok
INSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niaga= >ra/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET=3DSOLgnu
BINDISTMIN=3D/ho= >mes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
CM3CVSSERVER=3Dbirch.ele= >gosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvs
BINDISTMIN_NAME= >=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN=3D/homes/hosking/work/cm3-m= >in-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.elegosoft.com..
ssh birch= >.elegosoft.com ok
Building cm3.
Tinderbox Tree:  =3B =3B"cm3"= >
Buildname:  =3B =3B =3B =3B =3B =3B =3B"SOL= >gnu SunOS 5.10 niagara lastok-build"

creating log file /tmp/build-cm= >3-20081217-065704-RGaGCn/log.txt

---

checkout=2C compile and = >test of cm3 ...
2008.12.17 06:57:04 -- checkout in progress.
[start c= >heckout 2008.12.17 06:57:06]
cd /tmp/build-cm3-20081217-065704-RGaGCn/bu= >ild
cvs return value: 0
[end checkout 2008.12.17 07:16:18]
CHECKOU= >T_RETURN =3D 0
--
2008.12.17 07:16:21 -- compile in progress.
[sta= >rt compile 2008.12.17 07:16:21]
compile return value: 0
[end compile = >2008.12.17 08:01:31]
COMPILE_RETURN =3D 0
2008.12.17 08:01:38 -- test= >s in progress.
[start run-tests 2008.12.17 08:01:38]
cd /tmp/build-cm= >3-20081217-065704-RGaGCn/build
[end run-tests 2008.12.17 08:01:38]
TE= >STS_RETURN =3D 0
2008.12.17 08:01:38 -- checkout=2C compile and test run= > done.

---

removing build tree /tmp/build-cm3-20081217-065704= >-RGaGCn ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/nia= >gara-*

cleaning regression test log files...
/homes/hosking/tmp/c= >m3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking= >/tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3t= >ests-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extrac= >t

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-m= >in-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-= >report-SOLgnu-*.html

done.

PAN>

V>

body> >= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_-- From jay.krell at cornell.edu Thu Dec 18 13:25:48 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 12:25:48 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: > Do Cygwin users really care about all that stuff? Could be that some do, some don't. > The point of using Modula-3 on Cygwin But is "Cygwin" the point? Would Interix or UWin suffice? Maybe. Or just accepting forward slashes and providing a 64bit longint? Maybe. And what does "Cygwin" mean?It is many things. Maybe the point is to use X Windows? Not likely imho, but maybe. I hadn't considered this at first myself, but someone (Olaf?) asked for it. Or pthreads? Again not likely, but maybe. Or LONGINT? Maybe. Or Posix paths? ie. form and run command lines from bash. This seems to be the most popular reason. The "run" part, not a problem, the "form" part somewhat related to the path issue. Or to use a "super" optimizing compiler instead of just a fast "moderately" optimizing compiler? Or to use something you have more source to? i.e. gcc/ld/newlib. Or to interop with gcc-specific code? You can use a mix of compilers and interop, Cygwin, Visual C++, and the integrated backend all use the same object file format, but maybe it is easier to use one toolset more, than to mix. Or to use forward slashes in paths? You can do that with regular NT386 now, as you always could with the kernel32.dll functions. "Posix paths" vs. "forward slashes" not entirely the same thing, but almost. Unfortunately Win32 is not consistent here, like the file open dialogs and I think shlwapi.dll don't accept forward slashes. Arguably bugs. Or to interop with open/read/close/stat? You could always already do that too. Those functions do all exist in msvcr*.dll. Or to interop with very Berkeley socket-specific code? Maybe. I think Windows sockets are very very similar to Berkely/Posix sockets, but not quite the same. For example, I think an "fd set" is an array of opaque pointers, not a bit set. Or to link (again, interop) with code calling fork? Or because you want a better cross-build story? That is, you can build/run the Cygwin runtime, compiler, and linker on any other host, whereas the NT386 linker is only runnable on NT386. Or because you are familiar with gdb? Or due to issues around freedom of beer or liberty? Visual C++ has varied in its free-beer-ness. What is the point of the MinGWin project? I think it is about using autotools and gcc, but not sure. And producing binaries with "more flexible licensing" (no cygwin1.dll) One reason I got this stuff working is becauseI saw it as a shortcut to 64bit LONGINT support. I find Cygwin largely "cumbersome".Just usings its command line is slow.Slow to edit. gcc is slow.But the cross build story is very good.You know, I use Cygwin so that Windows can target Modula-3 to other systems. You know, to build m3cg. MinGWin I think suffices, but I got the impression at some point that Cygwin was better maintained than MinGWin. There are many factors.Modula-3 was already usable by Cygwin users, just as Visual C++ already is. The Modula-3 libraries shield you from almost all of this, if you stick to them.If you don't interop with much C code, if don't need a 64bit LONGINT, if thefast integrated backend generates efficient enough code for you, etc.The "external" interface really seems largely the form of paths. ...I /think/ most of the point to Cygwin to a lot of people, at some point, was to have a free beer C/C++ compiler. But again, this is only some people, and there are now other options. Digital Mars and Open Watcom are also pretty good. Digital Mars doesn't come with source. Open Watcom does. eBay provides pretty cheap stuff too. C/C++ compilers have lost their value as other programming languages (Java, JavaScript, C#, Perl, Python) have risen and sometimes been free (all free-beer, somewhat free-freedom/GPL). Somewhat due to the very "safety" issues that Modula-3 solves. Modula-3 I think was way ahead of its time. The world has mostly caught up, though Modula-3 is still a fairly unique hybrid, in terms of compiling directly to native code, and offering optional safety within one language. (C# also has optional safety within one language). If you only want safety and can sacrifice perf and/or generality and/or native code.. But again, there are many factors, many characteristics that define "Cygwin". - Jay> To: jay.krell at cornell.edu> Date: Wed, 17 Dec 2008 19:10:15 -0800> From: mika at async.caltech.edu> CC: jkrell at elego.de; m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > Do Cygwin users really care about all that stuff?> > The point of using Modula-3 on Cygwin is to turn a yucky Windows> machine as close as possible into a friendly Unix environment...> not to use a bunch of Microsoft tools and libraries---I would have> thought that if you wanted to use Visual X-Y-Z then you wouldn't> be using Cygwin but would be using the native Windows environment?> > So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> occasionally use on Windows uses Windows threads and has its own> compiler back-end. But it's kind of cool too and since it's not> really distinguishable from user-level threads and gcc I just think> "great" and get on with it. In fact I'm not even sure it uses> the integrated back end now that I think about it.> > I think what I'm saying is that almost no Cygwin users are terribly> interested in Microsoft interfaces, and I would bet most Cygwin> users aren't even aware of the existence of most Microsoft tools.> They (we) want as little to do with Windows as possible....> > Mika> > Jay writes:> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/plain; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >From: TonyTo: jay> >I would have thought of CygWin as a POSIX platform that just happens to hav=> >e Windows underneath. Did we go round this roundabout before?> >> >> >> >> >> >It's more complicated than that.> >It isn't just one bit.> >There's a few factors=2C that people care more and less about.> > GUI library -- Win32 or X> > thread library -- pthreads or Win32 (or user threads)=20> > file system path representation -- forward slashes or backward slashes => > This seems to be what people care most about -- to see forward slashes.> > C compiler -- Visual C++ cl or Cygwin/GNU gcc> > linker -- Visual C++ link or Cygwin/GNU ld=20> > C runtime -- msvcr*.dll or cygwin1.dll=20> > integrated backend or not The integrated backend is way faster=2C but=> > doesn't yet support 64bit LONGINT.> > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/=> >pdb> >Many of these factors are independent of each other=2C but not entirely.> >Compiler/linker directly lead to which debugger you can use=2C since they => >each use different formats.> >Well=2C you can use either debugger=2C but..without symbols..at least for s=> >ome modules.> >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa=> >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. => >This is probably fixable at least in one combination withan alias (ability => >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN=> >U ld understands aliases).> >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the=> >ir linker somehow accepts them.> >=20> >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by=> >te structs by value to __stdcall functions=2C the function name isn't mangl=> >ed correctly and linking fails.> >It is a bit of a combinatorial explosion.As the system stands=2C you can co=> >ntrol each bit in the config file=2C andmultiple combinations might work=2C=> > though I don't run any combinatorial tests.> >Three combinations are "identified" (named=2C spoken of) and have had some => >development testing.NT386=2C NT386MINGNU (which I think should be called I3=> >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN).> >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that=> >'s the name).Interix and UWIN being two other very viable Posix layers on W=> >indows.Interix has been renamed a few times. It is called "SFU" -- services=> > for Unix=2C and I don't know=2Clike SUA -- services for unix applications=> >=2C and I think there's another name.It is a free download from Microsoft f=> >or x86 Windows < Vista=2C and is an optional component in Vista.It has its => >pluses and minuses..not now.> >Anyway=2C in conversation here=2C one of the main things that people DO NOT=> > care aboutis which thread library is used.> >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h=> >ave used Win32 threads.> >And maybe never had any GUI?> >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres=> >h go at debugging it would pan out..but I'm really not too interested..> >I also have no interest in user threads. I believe they are never warranted=> >=2C excepton old systems that don't have kernel threads=2C like the hypothe=> >tical DJGPP port.> >I never brought up the issue of serial ports.I ASSUMED that almost nobody u=> >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem=> >ented using Cygwin or Win32=2C as long as they work.> >I ASSUME there is no interop issue=2C like getting an int fd from the seria=> >l portand passing it to select() or close=2C or getting a Win32 HANDLE and => >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly.> >I didn't look into if Cygwin is complete enough to implement them.I conside=> >r serial ports to be kind of out on the fringe..that it wouldn't be surpris=> >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui=> >lding minicom on some non-Linux system(s) recently and it failed miserably.=> > I don't know if there is even much Posix portability option for serial por=> >t programming.> >Sure..I debug over a serial port=2C and lately have machines I can't get vi=> >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us=> >e a serial consolefor initial install (recent achievement :) still haven't => >installed over network..)=2C but that's still imho a big step away from wri=> >ting code that uses the serial port.(And NetBSD doesn't support the local v=> >ideo on SGI O2=2C but I'm running OpenBSD there for now.)> >=20> >Anyway=2C there are a few ways to look at this issue.More and less trivial.> >A trivial view is: Get the filtering to work. Build serial on all platform=> >s but Cygwin. Or see if the Posix code will work.> > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p=> >rogrammers is not 0=2C but could be. I actually don't think anyone is using=> > Cygwin x Modula-3.> >Another view is: Get the Win32 serial code to work on Cygwin. The code is=> > already there=2C and buildable=2C but building in a system that also build=> >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio=> >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr=> >ovide the File interface=2C and FileWin32.m3 to provide just its own spec=> >ific FileWin32 interface. I think this is a simple transform of the code.> >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma=> >ybe I'll get the truncation I deserve. :)> > - Jay=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/html; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >> >> >> >
> >From: Tony
To: jay

> >
I would have thought of CygWin as a POSIX platform that just happens t=> >o have Windows underneath.  =3BDid we go round this roundabout before?<=> >/DIV>> >
>12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND=> >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS=> >E: separate">> >
>e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C=> >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S=> >PACING: normal=3B BORDER-COLLAPSE: separate"> >pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n=> >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B => >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN=> >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no=> >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T=> >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S=> >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet=> >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B=> > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate=> >"> >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT=> >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: => >separate"> >ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX=> >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO=> >LLAPSE: separate"> >0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0=> >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B=> >ORDER-COLLAPSE: separate">> >

 >V>
> >
> >
> FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T=> >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-=> >COLLAPSE: separate">> >
>>
It's more complicated than that.
> >
>>
It isn't just one bit.
> >
>>
There's a few factors=2C that people care more and less about.
> >
>>
 =3B GUI library =3B -- Win32 or X
> >
>>
 =3B thread library =3B -- pthreads or Win32 (or user threads)=> >
> >
>>
 =3B file system path representation -- forward slashes or backwar=> >d slashes
 =3B =3B =3B This seems to be what people care mos=> >t about -- to see forward slashes.
> >
>>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
> >
>>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
> >
>>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
> >
>>
 =3B integrated backend or not
 =3B =3B =3B The in=> >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT. >IV>> >
>>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi=> >suaStudio/CodeView/pdb
> >
>>
Many of these factors are independent of each other=2C but not entirel=> >y.
> >
>>
Compiler/linker directly lead to which debugger you can use=2C since >R> =3B they each use different formats.
> >
>>
Well=2C you can use either debugger=2C but..without symbols..at least => >for some modules.
> >
>>
The Cygwin C runtime and Visual C++ linker disagree
on the name of => >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual=> > C++ linker. This is probably fixable at least in one combination with
a=> >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't=> >
know if Cygwin/GNU ld understands aliases).
> >
>>
I have also found that Cygwin produces incorrect
import .libs=2C pe=> >rhaps their linker somehow accepts them.
> >
>> =3B
> >
>>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b=> >yte structs by value to __stdcall functions=2C the function name isn't mang=> >led correctly and linking fails.
> >
>>
It is a bit of a combinatorial explosion.
As the system stands=2C y=> >ou can control each bit in the config file=2C and
multiple combinations => >might work=2C though I don't run any combinatorial tests.
> >
>>
Three combinations are "identified" (named=2C spoken of) =3Band ha=> >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho=> >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called=> > I386_CYGWIN).
> >
>>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if=> > that's the name).
Interix and UWIN being two other very viable Posix la=> >yers on Windows.
Interix has been renamed a few times. It is called "SFU=> >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u=> >nix applications=2C and I think there's another name.
It is a free downl=> >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo=> >nent in Vista.
It has its pluses and minuses..not now.
> >
>>
Anyway=2C in conversation here=2C one of the main things that people D=> >O NOT care about
is which thread library is used.
> >
>>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi=> >ght have used Win32 threads.
> >
>>And maybe never had any GUI?
> >
>>
I tried getting pthreads to work=2C but gave up debugging it.
Perha=> >ps a fresh go at debugging it would pan out..but I'm really not too interes=> >ted..
> >
>>
I also have no interest in user threads. I believe they are never warr=> >anted=2C except
on old systems that don't have kernel threads=2C like th=> >e hypothetical DJGPP port.
> >
>>
I never brought up the issue of serial ports.
I ASSUMED that almost=> > nobody uses them=2C and anyone that does use them=2C doesn't
care if th=> >ey are implemented using Cygwin or Win32=2C as long as they work.
> >
>>I ASSUME there is no interop issue=2C like getting an int fd from the seri=> >al port
and passing it to select() or close=2C or getting a Win32 HANDLE=> > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c=> >ertainly.
> >
>>
I didn't look into if Cygwin is complete enough to implement them.
=> >I consider serial ports to be kind of out on the fringe..that it wouldn't b=> >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I => >tried building minicom on some non-Linux system(s) recently and it failed m=> >iserably. I don't know if there is even much Posix portability option for s=> >erial port programming.
> >
>>
Sure..I debug over a serial port=2C and lately have machines I can't g=> >et video
out of (Itanium) or even seem to have no video hardware (Sun)=> >=2C so I use a serial console
for initial install (recent achievement :)=> > still haven't installed over network..)=2C but that's still imho a big ste=> >p away from writing code that uses the serial port.
(And NetBSD doesn't => >support the local video on SGI O2=2C but I'm running OpenBSD there for now.=> >)
> >
>> =3B
> >
>>
Anyway=2C there are a few ways to look at this issue.
More and less=> > trivial.
> >
>>
A trivial view is:
 =3B Get the filtering to work. Build serial=> > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code => >will work.
> >
>> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria=> >l port programmers is not 0=2C but could be. I actually don't think anyone => >is using Cygwin x Modula-3.
> >
>>
Another view is:
 =3B Get the Win32 serial code to work on Cygw=> >in.
 =3B The code is already there=2C and buildable=2C but building => >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula=> >-3 rules=2C you end up with two implementations of the File interface.
&=> >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte=> >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci=> >fic FileWin32 interface.
 =3B I think this is a simple transform of => >the code.
> >
>>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor=> >ry.
Maybe I'll get the truncation I deserve. :)
> >
>>
 =3B- Jay
> >=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Thu Dec 18 21:46:26 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 18 Dec 2008 15:46:26 -0500 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: <494A6FFD.1E75.00D7.1@scires.com> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel about it and Modula-3. Mika has a point that if you want to create an environment on Windows that is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are looking for. On the other hand, I am in a camp of folks who don't mind using the Microsoft tools and want Modula-3 to work out of the box on Windows without having to add any other non-Microsoft stuff (like Cygwin). I think it is great for CM3 to be able to support both these camps! Thus, you will find me always advocating for keeping the "CM3 on Windows variant" where it will work on Windows using just the Microsoft tools. I am happy with the use of Windows threads and the native backend and use of the MS Visual Studio linker. Regards, Randy Coleburn >>> Mika Nystrom 12/17/2008 10:10 PM >>> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 22:21:07 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 18 Dec 2008 13:21:07 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> Message-ID: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Yes, I don't think the world is a terrible place because people like you exist :-) I just think that Cygwin users generally have modest requirements. They're just trying to get something---anything!---to work on Windows, and if they want to do anything that's really Windows-specific I think they/we understand they/we may have to make the effort to learn a bit about Windows and use the native environment. What it adds up to, in my opinion, is that one shouldn't screw things up in the native Windows port (or in the Unix ports) just to satisfy Cygwin users, who really don't care if Windows feature XYZ is available. How many people actually use Modula-3 on Cygwin? I do... and I just use it so I can run programs I developed on Unix on random Windows machines when the owners of those machines have made the bizarre (to me) decision to run Windows... If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y than it is now. For instance, the version I use has Windows file-locking semantics, i.e., default readers-and-writers locking on all open files, which can be quite annoying when porting from Unix. It also uses a different epoch for Time.T. Not sure if the current version has these features... Mika "Randy Coleburn" writes: >--=__Part6F4761B2.0__= >Content-Type: text/plain; charset=US-ASCII >Content-Transfer-Encoding: quoted-printable > >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >about it and Modula-3. >=20 >Mika has a point that if you want to create an environment on Windows that = >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >looking for. >=20 >On the other hand, I am in a camp of folks who don't mind using the = >Microsoft tools and want Modula-3 to work out of the box on Windows = >without having to add any other non-Microsoft stuff (like Cygwin). >=20 >I think it is great for CM3 to be able to support both these camps! >=20 >Thus, you will find me always advocating for keeping the "CM3 on Windows = >variant" where it will work on Windows using just the Microsoft tools. I = >am happy with the use of Windows threads and the native backend and use of = >the MS Visual Studio linker. >=20 >Regards, >Randy Coleburn > >>>> Mika Nystrom 12/17/2008 10:10 PM >>> >Do Cygwin users really care about all that stuff? > >The point of using Modula-3 on Cygwin is to turn a yucky Windows >machine as close as possible into a friendly Unix environment... >not to use a bunch of Microsoft tools and libraries---I would have >thought that if you wanted to use Visual X-Y-Z then you wouldn't >be using Cygwin but would be using the native Windows environment? > >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still >occasionally use on Windows uses Windows threads and has its own >compiler back-end. But it's kind of cool too and since it's not >really distinguishable from user-level threads and gcc I just think >"great" and get on with it. In fact I'm not even sure it uses >the integrated back end now that I think about it. > >I think what I'm saying is that almost no Cygwin users are terribly >interested in Microsoft interfaces, and I would bet most Cygwin >users aren't even aware of the existence of most Microsoft tools. >They (we) want as little to do with Windows as possible.... > > Mika > >Jay writes: >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >>Content-Type: text/plain; charset=3D"iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >>From: TonyTo: jay >>I would have thought of CygWin as a POSIX platform that just happens to = >hav=3D >>e Windows underneath. Did we go round this roundabout before? >> >> >> >> >> >>It's more complicated than that. >>It isn't just one bit. >>There's a few factors=3D2C that people care more and less about. >> GUI library -- Win32 or X >> thread library -- pthreads or Win32 (or user threads)=3D20 >> file system path representation -- forward slashes or backward slashes = > =3D >> This seems to be what people care most about -- to see forward slashes. >> C compiler -- Visual C++ cl or Cygwin/GNU gcc >> linker -- Visual C++ link or Cygwin/GNU ld=3D20 >> C runtime -- msvcr*.dll or cygwin1.dll=3D20 >> integrated backend or not The integrated backend is way faster=3D2C = >but=3D >> doesn't yet support 64bit LONGINT. >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView= >/=3D >>pdb >>Many of these factors are independent of each other=3D2C but not = >entirely. >>Compiler/linker directly lead to which debugger you can use=3D2C since = >they =3D >>each use different formats. >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least = >for s=3D >>ome modules. >>The Cygwin C runtime and Visual C++ linker disagreeon the name of = >__ImageBa=3D >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ = >linker. =3D >>This is probably fixable at least in one combination withan alias = >(ability =3D >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if = >Cygwin/GN=3D >>U ld understands aliases). >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps = >the=3D >>ir linker somehow accepts them. >>=3D20 >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing = >8 by=3D >>te structs by value to __stdcall functions=3D2C the function name isn't = >mangl=3D >>ed correctly and linking fails. >>It is a bit of a combinatorial explosion.As the system stands=3D2C you = >can co=3D >>ntrol each bit in the config file=3D2C andmultiple combinations might = >work=3D2C=3D >> though I don't run any combinatorial tests. >>Three combinations are "identified" (named=3D2C spoken of) and have had = >some =3D >>development testing.NT386=3D2C NT386MINGNU (which I think should be = >called I3=3D >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= From jay.krell at cornell.edu Thu Dec 18 23:12:12 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 22:12:12 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y Well, here I can cop-out and say: "Hey, it just uses Cygwin". Other than a possible avenue for the serial code. And other than threads. > Unix. It also uses a different epoch for Time.T. Not sure if I'm pretty sure that's not the case now. It very likely uses 31bit seconds since 1970, and will break around 2038. The hypothetical AMD64_CYGWIN would likely use 63bit seconds since 1970, so better. msvcr*.dll offers 63bits seconds since 1970, you configure it with a #define. (I say 31 or 63, because they are signed numbers probably, but negative numbers aren't considered valid. They really should be unsigned, and with a much earlier epoch...I realize files don't exist before these times, but it's nice to have one time format apply to more than just files, and there's certainly a legitimate need to represent earlier times..) Oh..well, yeah, I did name files foo.lib and not libfoo.a. File naming was another question that came up, along the lines of "What is Cygwin?". People said they didn't care. Similarly, executables named "foo" or "foo.exe"? .dlls named "foo.so" or "foo.dll". Cygwin actually uses foo.dll (or libfoo.dll, er..cygfoo.dll actually) and foo.exe. foo.so works fine, and "foo" works fine, at least on NT. bash will run it. cmd will run it if you add "." to %PATHEXT%. Apparently Win9x doesn't like it, and Cygwin historically supported Win9x (still today, but not for much longer, not with the next major release). So here you can see, that some typical Posix characteristics are not followed by Cygwin. Interix actually does use extensionless executables. You can see the evidence in taskmgr for example. I had it using foo.so (libfoo.so?) and "foo" for a bit, but since people didn't care, and Cygwin didn't do that, I put it back. It was pretty easy to go back and forth, like by fiddling with the "naming conventions" setting. (Which, it turns out, I think, was never really implemented correctly -- there are host and target conventions and I think they were confused.) Wait a sec, this does remind me. The cygfoo.dll convention has the benefit of being able to mix stuff in the same directory or along the same %PATH%. I thought that not useful, since .exes would still clash in name, but maybe? I will put serial back for non-Cygwin, really. And I'll fix SOLsun/gnu. Soon. - Jay> To: rcoleburn at scires.com> To: m3devel at elegosoft.com> Date: Thu, 18 Dec 2008 13:21:07 -0800> From: mika at async.caltech.edu> Subject: Re: [M3devel] serial/Cygwin> > Yes, I don't think the world is a terrible place because people like> you exist :-)> > I just think that Cygwin users generally have modest requirements.> They're just trying to get something---anything!---to work on> Windows, and if they want to do anything that's really Windows-specific> I think they/we understand they/we may have to make the effort to> learn a bit about Windows and use the native environment.> > What it adds up to, in my opinion, is that one shouldn't screw> things up in the native Windows port (or in the Unix ports) just> to satisfy Cygwin users, who really don't care if Windows feature> XYZ is available.> > How many people actually use Modula-3 on Cygwin? I do... and I> just use it so I can run programs I developed on Unix on random> Windows machines when the owners of those machines have made the> bizarre (to me) decision to run Windows...> > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> than it is now. For instance, the version I use has Windows> file-locking semantics, i.e., default readers-and-writers locking> on all open files, which can be quite annoying when porting from> Unix. It also uses a different epoch for Time.T. Not sure if> the current version has these features...> > Mika> > "Randy Coleburn" writes:> >--=__Part6F4761B2.0__=> >Content-Type: text/plain; charset=US-ASCII> >Content-Transfer-Encoding: quoted-printable> >> >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >about it and Modula-3.> >=20> >Mika has a point that if you want to create an environment on Windows that => >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >looking for.> >=20> >On the other hand, I am in a camp of folks who don't mind using the => >Microsoft tools and want Modula-3 to work out of the box on Windows => >without having to add any other non-Microsoft stuff (like Cygwin).> >=20> >I think it is great for CM3 to be able to support both these camps!> >=20> >Thus, you will find me always advocating for keeping the "CM3 on Windows => >variant" where it will work on Windows using just the Microsoft tools. I => >am happy with the use of Windows threads and the native backend and use of => >the MS Visual Studio linker.> >=20> >Regards,> >Randy Coleburn> >> >>>> Mika Nystrom 12/17/2008 10:10 PM >>>> >Do Cygwin users really care about all that stuff?> >> >The point of using Modula-3 on Cygwin is to turn a yucky Windows> >machine as close as possible into a friendly Unix environment...> >not to use a bunch of Microsoft tools and libraries---I would have> >thought that if you wanted to use Visual X-Y-Z then you wouldn't> >be using Cygwin but would be using the native Windows environment?> >> >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> >occasionally use on Windows uses Windows threads and has its own> >compiler back-end. But it's kind of cool too and since it's not> >really distinguishable from user-level threads and gcc I just think> >"great" and get on with it. In fact I'm not even sure it uses> >the integrated back end now that I think about it.> >> >I think what I'm saying is that almost no Cygwin users are terribly> >interested in Microsoft interfaces, and I would bet most Cygwin> >users aren't even aware of the existence of most Microsoft tools.> >They (we) want as little to do with Windows as possible....> >> > Mika> >> >Jay writes:> >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >>Content-Type: text/plain; charset=3D"iso-8859-1"> >>Content-Transfer-Encoding: quoted-printable> >>> >>> >>> >>From: TonyTo: jay> >>I would have thought of CygWin as a POSIX platform that just happens to => >hav=3D> >>e Windows underneath. Did we go round this roundabout before?> >>> >>> >>> >>> >>> >>It's more complicated than that.> >>It isn't just one bit.> >>There's a few factors=3D2C that people care more and less about.> >> GUI library -- Win32 or X> >> thread library -- pthreads or Win32 (or user threads)=3D20> >> file system path representation -- forward slashes or backward slashes => > =3D> >> This seems to be what people care most about -- to see forward slashes.> >> C compiler -- Visual C++ cl or Cygwin/GNU gcc> >> linker -- Visual C++ link or Cygwin/GNU ld=3D20> >> C runtime -- msvcr*.dll or cygwin1.dll=3D20> >> integrated backend or not The integrated backend is way faster=3D2C => >but=3D> >> doesn't yet support 64bit LONGINT.> >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView=> >/=3D> >>pdb> >>Many of these factors are independent of each other=3D2C but not => >entirely.> >>Compiler/linker directly lead to which debugger you can use=3D2C since => >they =3D> >>each use different formats.> >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least => >for s=3D> >>ome modules.> >>The Cygwin C runtime and Visual C++ linker disagreeon the name of => >__ImageBa=3D> >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ => >linker. =3D> >>This is probably fixable at least in one combination withan alias => >(ability =3D> >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if => >Cygwin/GN=3D> >>U ld understands aliases).> >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps => >the=3D> >>ir linker somehow accepts them.> >>=3D20> >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing => >8 by=3D> >>te structs by value to __stdcall functions=3D2C the function name isn't => >mangl=3D> >>ed correctly and linking fails.> >>It is a bit of a combinatorial explosion.As the system stands=3D2C you => >can co=3D> >>ntrol each bit in the config file=3D2C andmultiple combinations might => >work=3D2C=3D> >> though I don't run any combinatorial tests.> >>Three combinations are "identified" (named=3D2C spoken of) and have had => >some =3D> >>development testing.NT386=3D2C NT386MINGNU (which I think should be => >called I3=3D> >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Dec 18 23:03:06 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 18 Dec 2008 16:03:06 -0600 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: <494AC89A.4040601@wichita.edu> I agree that it is highly desirable to have both native windows and Cygwin variants of Modula-3. But I also think that it is unrealistic to try to support a complete or even partial Cartesian product of all the many separate options Jay enumerates. Without debating whether it might be desirable, it's just way too much work when there are other, more pressing things to do with our limited support time. Mix and match is just entirely unrealistic. By the same reasoning, I have given up on the similar ideal of making m3gdb handle a linked-together mix of code from different Modula-3 compilers: PM3, CM3, standalone backend, different versions of gcc-derived back end, etc. I advocate a Cygwin target that is as Unix-like as possible, in as many respects as possible, and a native Windows target that is as Windows-like as possible in as many respects as possible. No more. Mika Nystrom wrote: > Yes, I don't think the world is a terrible place because people like > you exist :-) > > I just think that Cygwin users generally have modest requirements. > They're just trying to get something---anything!---to work on > Windows, and if they want to do anything that's really Windows-specific > I think they/we understand they/we may have to make the effort to > learn a bit about Windows and use the native environment. > > What it adds up to, in my opinion, is that one shouldn't screw > things up in the native Windows port (or in the Unix ports) just > to satisfy Cygwin users, who really don't care if Windows feature > XYZ is available. > > How many people actually use Modula-3 on Cygwin? I do... and I > just use it so I can run programs I developed on Unix on random > Windows machines when the owners of those machines have made the > bizarre (to me) decision to run Windows... > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y > than it is now. For instance, the version I use has Windows > file-locking semantics, i.e., default readers-and-writers locking > on all open files, which can be quite annoying when porting from > Unix. It also uses a different epoch for Time.T. Not sure if > the current version has these features... > > Mika > > "Randy Coleburn" writes: >> --=__Part6F4761B2.0__= >> Content-Type: text/plain; charset=US-ASCII >> Content-Transfer-Encoding: quoted-printable >> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >> about it and Modula-3. >> =20 >> Mika has a point that if you want to create an environment on Windows that = >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >> looking for. >> =20 >> On the other hand, I am in a camp of folks who don't mind using the = >> Microsoft tools and want Modula-3 to work out of the box on Windows = >> without having to add any other non-Microsoft stuff (like Cygwin). >> =20 >> I think it is great for CM3 to be able to support both these camps! >> =20 >> Thus, you will find me always advocating for keeping the "CM3 on Windows = >> variant" where it will work on Windows using just the Microsoft tools. I = >> am happy with the use of Windows threads and the native backend and use of = >> the MS Visual Studio linker. >> =20 >> Regards, >> Randy Coleburn >> From jay.krell at cornell.edu Fri Dec 19 12:11:54 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 11:11:54 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <494AC89A.4040601@wichita.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> <494AC89A.4040601@wichita.edu> Message-ID: Pretty much agreed. The implementation is factored along the lines I describe, but I don't test the cross product. I think there might be one other interesting middle ground -- a target that uses the gcc backend but is otherwise Windows-like..oh, and probably GNU/gcc and GNU/ld, else you can't debug. This is analogous to the "MinGWin" project, which is real and works, it offers gcc, that can I believe build itself, but which generally produces binaries with no extra dependency or "strange" :) behavior. There is a config file for this called NT386MINGNU, though I think it should be called I386_MINGWIN. The system was able to build itself, and getting it to work was a subset of Cygwin -- they shared an important bug. This port can't use Trestle currently due to a minor bug in cm3cg. The benefits of this middle ground are that you get 64bit LONGINT already, and you might get highly optimized code instead of only moderately optimized code. Personally, I almost never ask gcc to optimize. You'll also probably get "AMD64_MINGW" before "AMD64_NT". Neither all that soon, but I know MinGWin/AMD64 is very far along or even done. (Other avenues to avoid updating the backend are C_NT and LLVM_NT, but these are so far just fantasies.) > The implementation is factored along the lines I describe And that is an exaggeration. It is mostly the config file that is factored such. Within the Modula-3 compiler, the only distinction is the "backend mode". The jmpbuf size is chosen I think to be the maximum of the two C runtimes. In the distant distant future, the compiler could also distinguish exception handling codegen strategies. There are several to chose from, like 3 or 4 for x86, 2 for the rest. - setjmp/longjmp -- as it stands now, or interoperable with gcc's similar - "elf/dwarf/whatever" -- range tables, stack walking, gcc supports this these days - NT/x86 style -- builds a linked list through a special per-thread/fiber location conceptually like setjmp/longjmp, but highly optimized. - NT/other -- basically like "elf", but perhaps the relevant data and runtime is different in the details - Jay> Date: Thu, 18 Dec 2008 16:03:06 -0600> From: rodney.bates at wichita.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > I agree that it is highly desirable to have both native windows and Cygwin> variants of Modula-3. But I also think that it is unrealistic to try to> support a complete or even partial Cartesian product of all the many> separate options Jay enumerates. Without debating whether it might be> desirable, it's just way too much work when there are other, more pressing> things to do with our limited support time.> > Mix and match is just entirely unrealistic. By the same reasoning, I> have given up on the similar ideal of making m3gdb handle a linked-together> mix of code from different Modula-3 compilers: PM3, CM3, standalone> backend, different versions of gcc-derived back end, etc.> > I advocate a Cygwin target that is as Unix-like as possible, in as many> respects as possible, and a native Windows target that is as Windows-like> as possible in as many respects as possible. No more.> > > > Mika Nystrom wrote:> > Yes, I don't think the world is a terrible place because people like> > you exist :-)> > > > I just think that Cygwin users generally have modest requirements.> > They're just trying to get something---anything!---to work on> > Windows, and if they want to do anything that's really Windows-specific> > I think they/we understand they/we may have to make the effort to> > learn a bit about Windows and use the native environment.> > > > What it adds up to, in my opinion, is that one shouldn't screw> > things up in the native Windows port (or in the Unix ports) just> > to satisfy Cygwin users, who really don't care if Windows feature> > XYZ is available.> > > > How many people actually use Modula-3 on Cygwin? I do... and I> > just use it so I can run programs I developed on Unix on random> > Windows machines when the owners of those machines have made the> > bizarre (to me) decision to run Windows...> > > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> > than it is now. For instance, the version I use has Windows> > file-locking semantics, i.e., default readers-and-writers locking> > on all open files, which can be quite annoying when porting from> > Unix. It also uses a different epoch for Time.T. Not sure if> > the current version has these features...> > > > Mika> > > > "Randy Coleburn" writes:> >> --=__Part6F4761B2.0__=> >> Content-Type: text/plain; charset=US-ASCII> >> Content-Transfer-Encoding: quoted-printable> >>> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >> about it and Modula-3.> >> =20> >> Mika has a point that if you want to create an environment on Windows that => >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >> looking for.> >> =20> >> On the other hand, I am in a camp of folks who don't mind using the => >> Microsoft tools and want Modula-3 to work out of the box on Windows => >> without having to add any other non-Microsoft stuff (like Cygwin).> >> =20> >> I think it is great for CM3 to be able to support both these camps!> >> =20> >> Thus, you will find me always advocating for keeping the "CM3 on Windows => >> variant" where it will work on Windows using just the Microsoft tools. I => >> am happy with the use of Windows threads and the native backend and use of => >> the MS Visual Studio linker.> >> =20> >> Regards,> >> Randy Coleburn> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Dec 19 19:25:52 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 18:25:52 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay From dabenavidesd at yahoo.es Sat Dec 20 01:09:41 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 20 Dec 2008 00:09:41 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Dear all: I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have. However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not? claim any more they are safe so that "fix" is not feasible. Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway?? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file. Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ),? that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation. There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that? Thanks in advance --- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/Cygwin Para: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25 > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Dec 20 08:19:02 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 19 Dec 2008 23:19:02 -0800 Subject: [M3devel] This disgusting TEXT business Message-ID: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Hello Modula-3ers, I have gone on the record before as not being very impressed by the Critical Mass implementation of TEXT. If you bother to read this email to the end, I will show you a simple program that runs 100 times slower when compiled with CM3 than the older PM3. TEXT is a very convenient built-in type, especially I think because it allows literals to be written with a lot less work than literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '\n' } Hence a lot of programs use TEXT liberally. Modula-3 also has a very syntactically convenient TEXT concatenation operator, &. I think it was Guy Steele who observed in one of the Lambda Papers that programmers tend to think syntactically concise notation is associated with efficient implementation. CM3 fails here, and sometimes horribly! I remember about five years ago we were writing a large number of VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" to CM3---I think one of us wanted to run the tools on a Macintosh. Most VLSI CAD tools really do nothing but process strings in various ways. Concatenation, splitting, those sorts of things. Think a large program written in a language without recursion but lots of hierarchical names. Compile unit x containing subunit y containing subunit z containing node n, giving name x.y.z.n . Well to make a long story short, the programmer with the Mac said that CM3 produced horribly slow-running binaries. Something with TEXTs, but we didn't investigate further. We gave up on migrating our group to CM3 and instead use PM3 to this day. The programmer with the Mac has since switched all his work to coding in C++ "in the style of Modula-3." And after seeing what I have seen now I cannot say I will voluntarily use CM3 for anything until we've got a solution. I would personally be happy to rip out everything that says Critical Mass TEXT on it and put back the SRC code. Here's the program I promised: MODULE Main; IMPORT Text, Fmt, IO, Time; CONST Count = 1000*1000; VAR checks := 0; start := Time.Now(); a : TEXT := ""; BEGIN FOR i := 0 TO 100 DO a := a & ":" & Fmt.Int(i) END; LOOP EVAL Text.Equal(a,a); INC(checks); IF checks > Count THEN IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / (Time.Now() - start)) & "\n"); start := Time.Now(); checks := 0 END END END Main. On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: virtual resident Text.Equals machine cycles size size per second compiler per char (approx) --------------------------------------------------------------- 11476K 2396K 13,274 CM3 770 1888K 960K 1,634,000 PM3 6.24 The culprit is TextCat.MyGetWideChars, which calls itself recursively. Here's a representative traceback: Starting program: /big/home/mika/test_text/FreeBSD4/test_text ^C Program received signal SIGINT, Interrupt. 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 111 TextCat.m3: No such file or directory. in TextCat.m3 (m3gdb) where #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) at Text.m3:66 #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 #10 0x08048b0d in Main (mode=1) at Main.m3:20 #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) at RTLinker.m3:122 #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at _m3main.mc:4 (m3gdb) Now you may object that perhaps people don't normally make TEXTs by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes they do. There's nothing to warn the user that that is a terribly bad thing to do. Well we know... we think.. it will lead to a lot of memory allocation overhead when you *make* the TEXT. But that it will infect the generated TEXT forevermore with this long chain of pointers is, to say the least, surprising. The reason I noticed this now is that I wrote a program that "watches" a directory of files (using FS.Iterate). It even uses a hash table to store the TEXTs. *Still* it is an incredible CPU hog (on Linux wth CM3). I think I just did path := dir & "/" & filename and that's enough to make it run 10-30 times slower than a PM3-compiled binary from the same source code (for suitably pathological choices of dir and filename). Does anyone have a better idea than just throwing away the CM3 TEXT implementation? Mika From hosking at cs.purdue.edu Sat Dec 20 09:26:40 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 20 Dec 2008 19:26:40 +1100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Hmm, are we just victims of poor implementation? Does anyone have the time to improve things? It would be possible to rip out CM3 TEXT and replace with PM3, but we'd lose WIDECHAR and WideText.T with that too. Not sure who that impacts. On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. > > If you bother to read this email to the end, I will show you a > simple program that runs 100 times slower when compiled with > CM3 than the older PM3. > > TEXT is a very convenient built-in type, especially I think because > it allows literals to be written with a lot less work than > > literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', > 'r', 'l', 'd', '!', '\n' } > > Hence a lot of programs use TEXT liberally. > > Modula-3 also has a very syntactically convenient TEXT concatenation > operator, &. > > I think it was Guy Steele who observed in one of the Lambda Papers > that > programmers tend to think syntactically concise notation is associated > with efficient implementation. > > CM3 fails here, and sometimes horribly! > > I remember about five years ago we were writing a large number of > VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" > to CM3---I think one of us wanted to run the tools on a Macintosh. > Most VLSI CAD tools really do nothing but process strings in various > ways. Concatenation, splitting, those sorts of things. Think a > large program written in a language without recursion but lots of > hierarchical names. Compile unit x containing subunit y containing > subunit z containing node n, giving name x.y.z.n . > > Well to make a long story short, the programmer with the Mac said > that CM3 produced horribly slow-running binaries. Something with > TEXTs, but we didn't investigate further. We gave up on migrating > our group to CM3 and instead use PM3 to this day. The programmer > with the Mac has since switched all his work to coding in C++ "in > the style of Modula-3." > > And after seeing what I have seen now I cannot say I will voluntarily > use CM3 for anything until we've got a solution. I would personally > be happy to rip out everything that says Critical Mass TEXT on it and > put back the SRC code. > > Here's the program I promised: > > MODULE Main; > IMPORT Text, Fmt, IO, Time; > > CONST > Count = 1000*1000; > > VAR > checks := 0; > start := Time.Now(); > a : TEXT := ""; > BEGIN > > FOR i := 0 TO 100 DO > a := a & ":" & Fmt.Int(i) > END; > > LOOP > EVAL Text.Equal(a,a); > INC(checks); > IF checks > Count THEN > IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / > (Time.Now() - start)) & "\n"); > start := Time.Now(); > checks := 0 > END > END > > END Main. > > On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo > E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: > > virtual resident Text.Equals machine cycles > size size per second compiler per char (approx) > --------------------------------------------------------------- > 11476K 2396K 13,274 CM3 770 > 1888K 960K 1,634,000 PM3 6.24 > > The culprit is TextCat.MyGetWideChars, which calls itself recursively. > Here's a representative traceback: > > Starting program: /big/home/mika/test_text/FreeBSD4/test_text > ^C > Program received signal SIGINT, Interrupt. > 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > 111 TextCat.m3: No such file or directory. > in TextCat.m3 > (m3gdb) where > #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) > at Text.m3:66 > #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 > #10 0x08048b0d in Main (mode=1) at Main.m3:20 > #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 > #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 > #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) > at RTLinker.m3:122 > #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at > _m3main.mc:4 > (m3gdb) > > Now you may object that perhaps people don't normally make TEXTs > by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes > they do. There's nothing to warn the user that that is a terribly > bad thing to do. Well we know... we think.. it will lead to a lot > of memory allocation overhead when you *make* the TEXT. But that > it will infect the generated TEXT forevermore with this long chain > of pointers is, to say the least, surprising. > > The reason I noticed this now is that I wrote a program that "watches" > a directory of files (using FS.Iterate). It even uses a hash table > to store the TEXTs. *Still* it is an incredible CPU hog (on Linux > wth CM3). I think I just did > > path := dir & "/" & filename > > and that's enough to make it run 10-30 times slower than a PM3- > compiled > binary from the same source code (for suitably pathological choices > of dir and filename). > > Does anyone have a better idea than just throwing away the CM3 TEXT > implementation? > > Mika From dragisha at m3w.org Sat Dec 20 09:00:54 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 09:00:54 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <1229760054.18256.134.camel@faramir.m3w.org> On Fri, 2008-12-19 at 23:19 -0800, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. If I ever saw an understatement here - then this is one :-) To make long story even shorter - what CM3 tried to do with their WIDECHAR bussiness and extra-fast concatenations was obsoleted very fast after (and some of it before) they put all that hell of code in their codebase. Whatever they thought to do - UTF-8 is doing with simple byte arrays, and string operations like array move are fastest non-trivial operations on CPUs since Z80. I hope we will see some end of it in times future. TIA. -- Dragi?a Duri? From dragisha at m3w.org Sat Dec 20 10:10:45 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 10:10:45 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Message-ID: <1229764245.18256.148.camel@faramir.m3w.org> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either slow, or produces results that make subsequent operations slow - is where problem is with this implementation. IMO, best solution would be to replace internal representation with UTF-8. For whom may be concerned with it - make some external widechar conversion routines available. That way - concat would be as fast as it can be made and other operations would be realistic - it is how almost everybody does their Unicode strings, after all. Almost everybody - excluding mobile industry and Microsoft :-), AFAIK. Compiler changes would make transparent conversion of literals. Everything else is already transparent with regard to internal representation. I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant to accept them as he did not believe C base routines were widespread enough. GNU world has no such reluctance. Everything is UTF8. If nobody can fill this job in next few weeks, I will probably have some time this winter. No promise, of course, but list will know :-). dd On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: > Hmm, are we just victims of poor implementation? Does anyone have the > time to improve things? It would be possible to rip out CM3 TEXT and > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > too. Not sure who that impacts. > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > > > Hello Modula-3ers, > > > > I have gone on the record before as not being very impressed by the > > Critical Mass implementation of TEXT. > > -- Dragi?a Duri? From jay.krell at cornell.edu Sat Dec 20 17:19:49 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 20 Dec 2008 16:19:49 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: My opinions.. UTF8 is a hack. It lets people do approximately nothing and claim they have done a lot. UTF8 is popular, but I'm still very skeptical of it. The best representation for Unicode is fixed size 16 bit characters, as Windows uses "everywhere". All the functions taking 8 bit characters are just thin wrappers over functions dealing with 16 bit characters. I believe NTFS stores 16 bit strings natively on disk. And so does FAT -- anything within a limited character set gets a regular old 8 bit 8.3 entry on disk. Anything "long", or with "other characters", or other forms like anything with two dots, I believe is stored as full 16 bit characters. In fact, I believe if you write an installable file system on Win9x, you deal with all 16 bit characters. What Windows does with 8 bit strings if of course much worse than UTF8. 8 bit characters on Windows are often considered "encoded in the current code page", with usually no record of the code page. I think probably the best thing to do with 8 bit characters is actually require them to be 7 bits. Or even 5 or 6 six bits, and conversion between them is just zero extension or 8 bit truncation. UTF8 is progress because it is, like, one universal code page. UTF8 is slightly muddied because Java defines it differently. Specifically I think there is an issue as to how to represent an embedded nul. There is also UTF7. I guess that's only useful in transports that aren't 8 bit clean. Is UTF a "transport" mechanism only, for an encoding to leave data in as much as possible? That is gray of course. "Entering" and "exiting" from "transport" is expensive when done in bulk, and avoided for perf. The best representation for a "string" is a length and an array of 16 bit characters, probably via a separate pointer, not rigth after the length. Arguably you need a 32 bit character since Unicde is, I believe, actually a 20 bit encoding. However here I'm willing to see things the other way. (ie: maybe hypocritical) 8 bit characters often suffice and it is probably worth being somewhat "dynamic". 8 bit bytes are also common, but should not be confused with characters and strings. Whatever functions any UTF8 code depends on, can be trivially implemented one's self, in portable C or portable Modula-3. How about.. I realized the Modula-3 code is "dynamic" and can deal with 8 or 16 bit characters. I didn't realize it concated strings via keeping around the pointers. This is actually a viable faster approach sometimes. They are called "ropes" in some contexts -- such as the SGI STL. I have to read the code, but..two obvious suggestions I am sympathetic to: 1) Leave things mostly alone, but add more functions people can call. Like String.Flatten, String.Cleanup, String.Seal. That would walk all the pointers and copy the data into one flat string. "Seal" is an exaggeration, since you could subsequently change the string. Problem with this approach of course is that any code that is slowed down, remains slowed down. You still have to do a little work to get back the perf. However, any code sped up by the current code remains sped up. 2) Always "flatten" upon concat. Remain dynamic in the representation, based on what characters are seen. Once a character above 127 is seen, the string is made wide. Any operation on a wide string that both returns a subset of it, AND has to anyway visit every character, can look for the opportunity to shorten it, IF a separate copy is made. Operations such as taking a prefix or suffix -- which can be made by just bumping the pointer or length, do NOT need to visit the characters and would not opportunistically narrow. Can taking a suffix like that work, with the garbage collector? Furthermore, the expression: foo & "a" & bar & "b" should be transformed, by the compiler, into one function call. If not currently, then "to do". This alone, with nothing else, might help a lot? Among the base-most string operations is not or shall not be "concat two strings", but rather "concat n strings". Hm. I think there's a third good option. Basically again looking to Windows. Windows has no one true string type..while internally Unicode is universal, all string functions are doubled up. For any given function foo, say for example CreateFile, there are actually two functions, CreateFileA and CreateFileW. To make a string literal in C or C++, you either say "foo" for an 8 bit string, or L"foo" for 16 bits. String length -- wcslen and strlen -- these names come from standard C and there are a bunch of them. pro: no existing code changes meaning con: no existing code changes meaning Anyone who needs unicode, needs to change their code, somehow. There is a static indirection mechanism. #ifdef UNICODE #define TCHAR WCHAR #define TEXT(a) L pasted to a #define CreateFile CreateFileW etc. #else #define TCHAR char #define a #define CreateFile CreateFileA etc. #endif You can write code to be portable either way. This was a dubious idea, because most code only compiles one way or the other. But it does let you migrate slowly -- keep 8 bit strings but write your code with the future in mind, and then later throw the switch. Personally I just hardcode everything WCHAR, CreateFileW, no illusions. Anyway, I bring this up to try to come up with a proposal along the lines of: - Take everything back to the way it was. - Offer some new way to support Unicode. but I'm not sure what the second part is. Is TEXT abstract enough that it can do double duty? What sort of interop is needed between code using narrow text and code using wide text? Must it be "automatic"? While I say UTF8 is a hack, I like something almost the same as Dragi?a says. Keep everything as 7 or 8 bit characters. Provide conversion routines. Make no assumptions internally that the characters are UTF8. Any internal string as array of char walking would just use individual bytes. Or a "decoding callback". ? Reject any text literal that isn't 7 or 8 bit clean. ? I don't know, I guess I'm very on the fence. Unicode is important and all, interop seems maybe important, it is hard to get anyone to change existing code..what CM3 did at least was seamless..in terms of no code change, well, er..perf change, and they barred direct access to the characters, which might still be good. Maybe you can ask for a direct pointer, of a specific type, and if you ask for the wrong type, you get NULL? ?? Now, this email contains no code to back it up. :) - Jay> From: dragisha at m3w.org> To: m3devel at elegosoft.com> Date: Sat, 20 Dec 2008 10:10:45 +0100> Subject: Re: [M3devel] This disgusting TEXT business> > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either> slow, or produces results that make subsequent operations slow - is> where problem is with this implementation.> > IMO, best solution would be to replace internal representation with> UTF-8. For whom may be concerned with it - make some external widechar> conversion routines available.> > That way - concat would be as fast as it can be made and other> operations would be realistic - it is how almost everybody does their> Unicode strings, after all. Almost everybody - excluding mobile industry> and Microsoft :-), AFAIK.> > Compiler changes would make transparent conversion of literals.> Everything else is already transparent with regard to internal> representation.> > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant> to accept them as he did not believe C base routines were widespread> enough. GNU world has no such reluctance. Everything is UTF8.> > If nobody can fill this job in next few weeks, I will probably have some> time this winter. No promise, of course, but list will know :-).> > dd> > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote:> > Hmm, are we just victims of poor implementation? Does anyone have the > > time to improve things? It would be possible to rip out CM3 TEXT and > > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > > too. Not sure who that impacts.> > > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote:> > > > > Hello Modula-3ers,> > >> > > I have gone on the record before as not being very impressed by the> > > Critical Mass implementation of TEXT.> > >> > -- > Dragi?a Duri? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Dec 20 19:20:12 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 19:20:12 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <1229797212.18256.157.camel@faramir.m3w.org> On Sat, 2008-12-20 at 16:19 +0000, Jay wrote: > My opinions.. > > UTF8 is a hack. > It lets people do approximately nothing and claim they have done a > lot. > UTF8 is popular, but I'm still very skeptical of it. I am very skeptical to skepticism without arguments. Extremelly skeptical when argument is "Windows does it...". > The best representation for Unicode is fixed size 16 bit characters, > as Windows > uses "everywhere". All the functions taking 8 bit characters are > just thin > wrappers over functions dealing with 16 bit characters. And both these facts/argument/whatever sounds scary, at least for me, me being a person of Modula-2/Modula-3 upbringing :-). UTF-8 has done everything it promised. UTF-8 is everywhere, except in Windows and mobile industry. This I/O conversion routines will do when cooperating with these worlds. As you pointed out - 16bit is not enoough for Unicode. UTF-8 covers all Unicode characters. If only this one is argument for Modula-3 Unicode representation in 2009 - it is enough for me. -- Dragi?a Duri? From roland.illig at gmx.de Sat Dec 20 19:38:01 2008 From: roland.illig at gmx.de (Roland Illig) Date: Sat, 20 Dec 2008 19:38:01 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <494D3B89.3020802@gmx.de> Dragi?a Duri? schrieb: > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. I object. UTF-8 is a good external representation format, but in memory, it is far more efficient to let all characters of a string have the same size. Java, for example, has defined a character to be "what fits into 16 bits". That's good enough in many situations. One of the situations it fails is when you want to represent Domino Tiles or Mahjong Tiles as characters. To give you some arguments against UTF-8 as in-memory representation: * How fast can you calculate the length of a string? (With length I mean the number of characters, not the memory size needed for storing them.) * Where does substring(5) start? Where does substring(10000) start, and how long does it take to find out? An argument for the UTF-8 representation is that it saves memory -- one byte for every ASCII character. Is this argument still as important as it has been 40 years ago? Roland From wagner at elegosoft.com Sat Dec 20 20:27:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 20 Dec 2008 20:27:55 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Hi everybody, I haven't followed all the discussions recently, but think it may help to comment on a few things here. IIRC the main reason for the current TEXT implementation was to become compatible with Java. Java strings and M3 strings should look the same for the embedded JVM written in M3. AFAIK this JVM has never been made open source nor is there any intention to do it. Still it may be a good idea to adhere to the TEXT representation if we ever intend to combine Java and M3 in one runtime. I don't think the implementation of the combined 8/16 bit TEXTs is very mature. I think that some improvements could be made. Has anybody looked into the details of the implementation? As it is already pointed out on the old CM3 notes on the web pages, some things are slower while others are now faster. I agree that the implementation is rather suboptimal for some common use cases. We'd need a set of use cases and tests agreed upon to really compare implementations though. Just one test for cat is not enough IMO. This should be the first step for an improvement. I'd also not object to a separation of the 8 and 16 bit implementations, but the type WIDECHAR etc. should remain at least for compatibility. We could perhaps even add some automatic conversion then between TEXT and WIDETEXT types (though this would not be in the M3 spirit :-) If we follow this way, the standard TEXTs could be replaced by the old PM3 implementation again. I'd also like to point out that much of the speed of using the old implementation comes by using the internal TextF interface, which exposes the internal TEXT representation as an array of CHAR. I'm not sure if this is really a good idea. Replacing direct array access by Text.GetChar was one of the main adaptions needed when converting all the standard packages to the CM3 compiler for the first release. I'd rather object to exposing this interface again. We could even define the standard TEXT representation as UTF8 in CM3 instead of the standard ISO Latin1 code set, which seems a bit euro- centric ;-) I'd support this idea, once we have proper UTF8 support in CM3 (which we currently haven't if I am not mistaken, see below). Quoting Dragi?a Duri? : > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. > Everything else is already transparent with regard to internal > representation. I am not sure that it is as easy as that. I'd like to see a proper update of the specification before we implement such a change. > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. No, the problem was that what I got couldn't even be compiled on FreeBSD and Solaris. We should not rely on some external GNU libraries, but we need to implement the UTF8 functions in CM3. Has anybody done that yet? > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). I'd be in favour of any improvements of the TEXT implementation and UTF8 support in CM3! > dd > > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >> Hmm, are we just victims of poor implementation? Does anyone have the >> time to improve things? It would be possible to rip out CM3 TEXT and >> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >> too. Not sure who that impacts. I'd be careful here, too. I think we need to understand the problems and the impacts of changes better before we rip out much code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From martinbishop at bellsouth.net Sat Dec 20 22:45:36 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sat, 20 Dec 2008 15:45:36 -0600 Subject: [M3devel] CM3 Source "pure"? Message-ID: <494D6780.6060701@bellsouth.net> I was looking through the CM3 sources out of boredom and noticed that there seems to be no C code, not even for runtime stuff? Are the CM3 sources "pure" in that they don't use any C code directly? (I did see some uses of M3toC and such). From hosking at cs.purdue.edu Sat Dec 20 23:05:18 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 21 Dec 2008 09:05:18 +1100 Subject: [M3devel] CM3 Source "pure"? In-Reply-To: <494D6780.6060701@bellsouth.net> References: <494D6780.6060701@bellsouth.net> Message-ID: <4D272586-0C33-4B7B-B8E6-7BD6A34C1B23@cs.purdue.edu> On 21 Dec 2008, at 08:45, Martin Bishop wrote: > I was looking through the CM3 sources out of boredom and noticed that > there seems to be no C code, not even for runtime stuff? Precisely, Modula-3 is a systems programming language just as capable as C. To that end, a goal of any language implementation is to "eat your own dog food" and implement as much of the language in itself as possible. > Are the CM3 sources "pure" in that they don't use any C code directly? > (I did see some uses of M3toC and such). There are some C sources that adapt to C library and system functions, but generally use of C is discouraged in the Modula-3 implementation. From darko at darko.org Sun Dec 21 00:08:57 2008 From: darko at darko.org (Darko) Date: Sun, 21 Dec 2008 08:08:57 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: The right way to do this, IMNSHO is to not assume any particular representation of TEXT values and create an implementation interface that allows implementations of multiple text representations, much like Rd and Wr don't make many assumptions about how data is actually stored or retrieved. This allows: - backward compatibility with the existing scheme - people's favourite representations (I personally don't think UTF-8 or UTF-16 are much chop as a single solution) - compatibility with external representation requirements (not having to convert each time you call an external API) - optimisation for particular application implementations The complication is with allowing representations to change at runtime and (automatic) conversion between representations, which I don't think adds too much complexity for the possible benefit. Also we need a proper UNICODE type that can contain only valid Unicode characters since WIDECHAR can't represent them all in one value. On 21/12/2008, at 4:27 AM, Olaf Wagner wrote: > Hi everybody, > > I haven't followed all the discussions recently, but think it may help > to comment on a few things here. > > IIRC the main reason for the current TEXT implementation was to become > compatible with Java. Java strings and M3 strings should look the same > for the embedded JVM written in M3. AFAIK this JVM has never been made > open source nor is there any intention to do it. > Still it may be a good idea to adhere to the TEXT representation if > we ever intend to combine Java and M3 in one runtime. > > I don't think the implementation of the combined 8/16 bit TEXTs is > very mature. I think that some improvements could be made. Has anybody > looked into the details of the implementation? As it is already > pointed > out on the old CM3 notes on the web pages, some things are slower > while > others are now faster. I agree that the implementation is rather > suboptimal > for some common use cases. We'd need a set of use cases and tests > agreed > upon to really compare implementations though. Just one test for cat > is > not enough IMO. This should be the first step for an improvement. > > I'd also not object to a separation of the 8 and 16 bit > implementations, > but the type WIDECHAR etc. should remain at least for > compatibility. We could perhaps even add some automatic conversion > then between TEXT and WIDETEXT types (though this would not be in the > M3 spirit :-) If we follow this way, the standard TEXTs could be > replaced by the old PM3 implementation again. > > I'd also like to point out that much of the speed of using the old > implementation comes by using the internal TextF interface, which > exposes the internal TEXT representation as an array of CHAR. I'm > not sure if this is really a good idea. Replacing direct array access > by Text.GetChar was one of the main adaptions needed when converting > all the standard packages to the CM3 compiler for the first release. > I'd rather object to exposing this interface again. > > We could even define the standard TEXT representation as UTF8 in CM3 > instead of the standard ISO Latin1 code set, which seems a bit euro- > centric ;-) I'd support this idea, once we have proper UTF8 support > in CM3 (which we currently haven't if I am not mistaken, see below). > > Quoting Dragi?a Duri? : > >> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is >> either >> slow, or produces results that make subsequent operations slow - is >> where problem is with this implementation. >> >> IMO, best solution would be to replace internal representation with >> UTF-8. For whom may be concerned with it - make some external >> widechar >> conversion routines available. >> >> That way - concat would be as fast as it can be made and other >> operations would be realistic - it is how almost everybody does their >> Unicode strings, after all. Almost everybody - excluding mobile >> industry >> and Microsoft :-), AFAIK. >> >> Compiler changes would make transparent conversion of literals. >> Everything else is already transparent with regard to internal >> representation. > > I am not sure that it is as easy as that. I'd like to see a proper > update of the specification before we implement such a change. > >> I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was >> reluctant >> to accept them as he did not believe C base routines were widespread >> enough. GNU world has no such reluctance. Everything is UTF8. > > No, the problem was that what I got couldn't even be compiled on > FreeBSD and Solaris. We should not rely on some external GNU > libraries, > but we need to implement the UTF8 functions in CM3. Has anybody done > that yet? > >> If nobody can fill this job in next few weeks, I will probably have >> some >> time this winter. No promise, of course, but list will know :-). > > I'd be in favour of any improvements of the TEXT implementation and > UTF8 support in CM3! > >> dd >> >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have >>> the >>> time to improve things? It would be possible to rip out CM3 TEXT >>> and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > > I'd be careful here, too. I think we need to understand the problems > and the impacts of changes better before we rip out much code. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From mika at async.caltech.edu Sun Dec 21 01:02:11 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 20 Dec 2008 16:02:11 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Sat, 20 Dec 2008 20:27:55 +0100." <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Olaf Wagner writes: ... >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have the >>> time to improve things? It would be possible to rip out CM3 TEXT and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > >I'd be careful here, too. I think we need to understand the problems >and the impacts of changes better before we rip out much code. > I do agree. My comments about simply putting back the PM3 code weren't meant to be taken seriously. I myself have always disliked this "internationalization" business. I remember my father used to have to prefix prices in Telexes to England with an o-with-umlaut (the o-with-umlaut on his Swedish Siemens printed as a lira on a British machine). And there's always been something vaguely Socialistic about it: I don't mean the fact that I had to use A-umlaut and Scandinavian-A-with-circle for square brackets on the C-64, but that on the very first computer I ever used, the Swedish Luxor ABC-80 (Zilog Z80 based), the Swedish standard of the time called for the dollar sign to be replaced by a small "sun symbol" pronounced "sol" in Swedish. BASIC strings were A-sol and B-sol, etc. I do not believe the "sol" is any more useful than the "dollar" in Sweden (probably less, especially in the winter); I have a feeling that Swedish standards bodies at the time were simply trying to show their sympathies for the money-less regime of the Khmer Rouge. But that being said, I realize there are people out there who might want non-ASCII in their strings. I think if the CM3 implementation does that well it might be worth fixing... not sure that UTF encoding is any simpler, overall...? A peek at Text.Equal sheds some light on the problem: PROCEDURE Equal (t, u: T): BOOLEAN = VAR info_t, info_u: Info; BEGIN t.get_info (info_t); u.get_info (info_u); IF (info_t.length # info_u.length) THEN RETURN FALSE; END; IF (info_t.length = 0) THEN RETURN TRUE; END; IF (info_t.start = NIL) OR (info_u.start = NIL) OR (info_t.wide # info_u.wide) THEN RETURN EqualBuf (t, u, info_t.length); ELSIF NOT info_t.wide THEN RETURN String8.Equal (info_t.start, info_u.start, info_t.length); ELSE RETURN String16.Equal (info_t.start, info_u.start, info_t.length); END; END Equal; Under CM3, Text.Cat/RTHooks.Concat (&) returns a special subtype of string, viz., TextCat.T, which has the following for get_info: PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; The NIL for info.start is what pushes the code into EqualBuf, which compares the strings character by character, but asks for each character via the get_wide_chars method, which calls itself recursively for each concatenated component. info.start = NIL is, in short, a flag that means "non-standard TEXT, use heavily object-oriented character-by-character routines". It seems to me this implementation could be sped up a great deal by doing some simple micro-optimizations. Let me ask a very simple question... is it possible to sub-type TEXT in CM3? (Is such an interface exported?) If not, it would be quite easy to de-object-orient this code and get most of the PM3 performance back, I think (we'd have to flatten the strings a bit too). If the code can be subtyped by clients outside of the text package it would be necessary to sort out what types of overrides should be supported, and what should happen if someone, for instance, subtypes a TextCat.T to have a different method for getting characters... I will try to get some time to look at this myself over the holidays. Note to self and others: m3gdb is not completely useful here. If you look at the structure of a TextCat.T, you see that fields a and b are TEXTs, but m3gdb doesn't really let you look inside them, since, well, they are TEXTs and printed as such. (You don't get to see the whole gory mess of pointers.) Is there a way to turn off this (normally very helpful) m3gdb feature? Mika From hendrik at topoi.pooq.com Sun Dec 21 21:40:03 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Sun, 21 Dec 2008 15:40:03 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081221204002.GA26711@topoi.pooq.com> On Sat, Dec 20, 2008 at 10:10:45AM +0100, Dragi?a Duri? wrote: > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. UTF-8 is indeed designed so that a lot of operations can be oerformed directly on the bytes of UTF-8. For example, if you sort UTF-8 strings based on the 8-bit unsigned values of their bytes, you get the same order as if you sorted them based on the integer values of their Unicode characters. As for delimiting strings based on character in them, such as single-byte spaces, you can just process it as a byte stream. If you want to find a character that's, say, three UTF8 bytes, you can do ordinary byte-string search for a three-byte string -- UTF-8 has self-delimiting characters. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. I worked on a C compiler long ago, and read the standard in great detail to determine that quoted strings contained characters, not bytes. As a result, the compiler we wrked on had a bug when it came to handling Korean characters when working in a Korean environ -- one of the Korean two-byte characters happened to contain a null byte, and the compiler treated it internally as end-of-string. Of course, we fixed, that, using locale-dependent character-parsing routines. Now Unicode doesn't even have that problem -- the only character containing a null byte is the null character, and it's a single null byte. (Java messes this up by using a different encoding for the null character, so that they can treat their compactly-encoded strings as being null-terminated. But they don't claim to used UTF-8 either, though they do almost use it. The encoding of the null character they use is such that almost any UTF-8 DEcoder will decode it as being a NULL character, unless it is specifically designed to check for it as an error condition. Just look ar the high-oprder bits of the bytes and do your shifts and you'll be OK.) I routinely use UTF-8 as the normal character code in all the software I write, and I almost never have an occasion when library-provided encoding and decoding functions are of much use. If I care much about the syntax, I'm using a parser anyway, and a parser can just as easlily parse bytes with the high bit set as bytes without. -- hendrik > Everything else is already transparent with regard to internal > representation. > > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. > > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). What's the job in question? Code that handles UTF-8? New data structures just for UTF-8 strings? Or code that just processes ordinary strings and does special processing when indicated on bytes with the high bit set? -- hendrik From dragisha at m3w.org Sun Dec 21 11:33:18 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 21 Dec 2008 11:33:18 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <494D3B89.3020802@gmx.de> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> Message-ID: <1229855598.18256.168.camel@faramir.m3w.org> Most important argument for UTF-8 - IMO - would be it's completeness. 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit storage magic instead? Accepting 2 byte would be same mistake once over. Once Latin1 was whole world. Is it 2byte now? As for "length" argument - it is cacheable data. As our TEXT is object type, we can do such things. As for "substr", it is sound counter argument. Internal optimizations are possible, of course. For lengthy strings, we can implement it, and efficiently. Once internal representation of TEXT was possible to pass to C without conversion. With UTF8 - it's possible again. Java is not world around us - it's C. And again - it's not it's storage efficiency that coounts - it's completeness. On Sat, 2008-12-20 at 19:38 +0100, Roland Illig wrote: > Dragi?a Duri? schrieb: > > IMO, best solution would be to replace internal representation with > > UTF-8. For whom may be concerned with it - make some external widechar > > conversion routines available. > > > > That way - concat would be as fast as it can be made and other > > operations would be realistic - it is how almost everybody does their > > Unicode strings, after all. Almost everybody - excluding mobile industry > > and Microsoft :-), AFAIK. > > I object. UTF-8 is a good external representation format, but in memory, > it is far more efficient to let all characters of a string have the same > size. Java, for example, has defined a character to be "what fits into > 16 bits". That's good enough in many situations. One of the situations > it fails is when you want to represent Domino Tiles or Mahjong Tiles as > characters. > > To give you some arguments against UTF-8 as in-memory representation: > > * How fast can you calculate the length of a string? (With length I mean > the number of characters, not the memory size needed for storing them.) > > * Where does substring(5) start? Where does substring(10000) start, and > how long does it take to find out? > > An argument for the UTF-8 representation is that it saves memory -- one > byte for every ASCII character. Is this argument still as important as > it has been 40 years ago? > > Roland -- Dragi?a Duri? From stsp at elego.de Sun Dec 21 16:40:15 2008 From: stsp at elego.de (Stefan Sperling) Date: Sun, 21 Dec 2008 15:40:15 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <20081221154015.GA6543@jack.stsp.name> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: > The right way to do this, IMNSHO is to not assume any particular > representation of TEXT values and create an implementation interface > that allows implementations of multiple text representations, much like > Rd and Wr don't make many assumptions about how data is actually stored > or retrieved. Such an interface may be needed for UTF-8 alone already, anyway, because within UTF-8 there is in some cases more than one way to store what amounts to the same data to a human user. In Subversion, from the beginning everyone agreed that the internal encoding for all strings would be UTF-8. Most Subversion APIs expect data in UTF-8. Strings (e.g. filenames) in the repository are stored in UTF-8, etc. Great! Will work in all countries! Right? Yes, but not on all operating systems if you're not careful! It did not occur to anyone at the time that there are characters which in UTF-8 have more than one representation (codepoints) in a byte stream. For example, an u with umlaut can be encoded as two bytes or a single byte: 2 bytes: [u | the previous character has an umlaut ] This is called "normal form decomposed". 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) This is called "normal form composed". If you want to be portable, as CM3 and Subversion both try to be, you have to consider that some operating systems may return your filenames in a different encoding then you stored it in: -------- Accepts Gives back MacOS X * NFD(*) Linux * Windows * Others ? ? *) There are some remarks to be made regarding full or partial NFD here, but the essential thing is: If you send in NFC, don't expect it back! -------- quoted from: http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames which is worth a read for more details if you're interested. In Subversion, this is a real problem for Mac users, because two filenames which only differ in their NFC/NFD encoding look exactly the same to the user (an u umlaut is printed), while the byte streams do not match ("We're sorry, but your file x does not exist in the repository!", where x looks just like a file that is clearly visible in the repository listing :) Subversion's problem now is that there are repositories out there using filenames in either NFC, NFD, or mixed, and there is no good way to reconcile the mess while staying backwards compatible with existing clients, servers, working copies and repositories. So Mac users are told to only use ASCII characters in their filenames to prevent the problem (many users, especially users who are not programmers, who store their photos or their entire home directory or whatever in Subversion, are not happy about this). This problem may not matter as much in case of CM3, but anyone implementing UTF-8 support for CM3 should be aware of this issue and not repeat the mistake the Subversion developers made at the time! With UTF-8, do not rely on a filename to retain its encoding as you passed it to the OS when requesting the filename from the OS again. CM3 should pick either NFD or NFC as internal UTF-8 encoding, for filenames only, or for all strings, whichever makes more sense. And then stick to it, converting input/output as needed. Abstracting this problem away using a nice interface would probably be the cleanest solution. Stefan From jay.krell at cornell.edu Sun Dec 21 18:51:14 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:51:14 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: This is an issue with Unicode, not UTF8, right? ie: a 16 or 20 or 32 bit encoding has the same problem, right? search the web for "unicode precomposed": http://en.wikipedia.org/wiki/Precomposed_character http://wikisource.org/wiki/Unicode_precomposed_characters or "unicode precomposed apple": http://developer.apple.com/jp/qa/qa2001/qa1235.html http://developer.apple.com/qa/qa2001/qa1235.html "When working within Mac OS you will find yourself using a mixture of precomposed and decomposed Unicode. For example, HFS Plus converts all file names to decomposed Unicode, while Macintosh keyboards generally produce precomposed Unicode. This isn't a problem as long as you use system-provided APIs to process text. Apple's APIs correctly handle both precomposed and decomposed Unicode. However, you may need to convert to precomposed Unicode when you interact with other platforms. For example, the following are all valid reasons why you might want to convert to precomposed Unicode. If you implement a network protocol which is defined to use precomposed Unicode. When creating a cross-platform file (or volume) whose specification dictates precomposed Unicode. If you incorporate a large body of cross-platform code into your application, where that code is expecting precomposed Unicode. " http://www.unicode.org/unicode/reports/tr15/index.html I need to read these.. - Jay> Date: Sun, 21 Dec 2008 15:40:15 +0000> From: stsp at elego.de> To: darko at darko.org> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] This disgusting TEXT business> > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote:> > The right way to do this, IMNSHO is to not assume any particular > > representation of TEXT values and create an implementation interface > > that allows implementations of multiple text representations, much like > > Rd and Wr don't make many assumptions about how data is actually stored > > or retrieved.> > Such an interface may be needed for UTF-8 alone already, anyway,> because within UTF-8 there is in some cases more than one way> to store what amounts to the same data to a human user.> > In Subversion, from the beginning everyone agreed that the internal> encoding for all strings would be UTF-8. Most Subversion APIs expect> data in UTF-8. Strings (e.g. filenames) in the repository are stored> in UTF-8, etc. Great! Will work in all countries! Right?> > Yes, but not on all operating systems if you're not careful!> It did not occur to anyone at the time that there are characters> which in UTF-8 have more than one representation (codepoints) in a> byte stream. For example, an u with umlaut can be encoded as two> bytes or a single byte:> > 2 bytes: [u | the previous character has an umlaut ]> This is called "normal form decomposed".> > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :)> This is called "normal form composed".> > If you want to be portable, as CM3 and Subversion both try to be,> you have to consider that some operating systems may return your> filenames in a different encoding then you stored it in:> > --------> Accepts Gives back> MacOS X * NFD(*)> Linux * > Windows * > Others ? ?> > > *) There are some remarks to be made regarding full or partial> NFD here, but the essential thing is: If you send in NFC, don't> expect it back!> -------- quoted from:> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames> which is worth a read for more details if you're interested.> > In Subversion, this is a real problem for Mac users, because> two filenames which only differ in their NFC/NFD encoding> look exactly the same to the user (an u umlaut is printed),> while the byte streams do not match ("We're sorry, but your> file x does not exist in the repository!", where x looks just> like a file that is clearly visible in the repository listing :)> > Subversion's problem now is that there are repositories out> there using filenames in either NFC, NFD, or mixed, and there> is no good way to reconcile the mess while staying backwards> compatible with existing clients, servers, working copies and> repositories. So Mac users are told to only use ASCII characters> in their filenames to prevent the problem (many users, especially> users who are not programmers, who store their photos or their> entire home directory or whatever in Subversion, are not happy> about this).> > This problem may not matter as much in case of CM3, but anyone> implementing UTF-8 support for CM3 should be aware of this issue> and not repeat the mistake the Subversion developers made at the> time! With UTF-8, do not rely on a filename to retain its encoding> as you passed it to the OS when requesting the filename from the> OS again.> > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for> filenames only, or for all strings, whichever makes more sense.> And then stick to it, converting input/output as needed.> > Abstracting this problem away using a nice interface would probably> be the cleanest solution.> > Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 21 18:57:00 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:57:00 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <465668.39592.qm@web28601.mail.ukl.yahoo.com> References: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Message-ID: The real world is unsafe.Safe areas are created by unsafe code, which must be, to a certain extent, bug free, in order to provide as safe as advertised. I don't believe what I added is any more untrusted/unsafe than what was there before.Well, maybe in a minor way. m3core has "lots" of "external" functions, which are called by "lots" of functions at higher levels..ok, maybe not "lots", but if you look at the m3core/src/unix directories that I have worked on -- cygwin, openbsd, amd64_linux, linux-common -- this is a fairly minimal set of stuff that is being used by preexisting code. A LITTLE bit I added beyond what was there. If you look at all the other directories, you'll find lots more external stuff, most of which is not used within the CM3 codebase (but could be used by folks outside). It is true I may have introduced external where it wasn't previously, but I don't think it matters.It is prevalent one level down. Moving some of it one level up is ok I believe. Non-Posix does potentially have a place in .c files.It depends.Basically, as I see things, a small amount of #ifdef is ok.A lot of #ifdef becomes hard to read, and at some point, you split the files into separate files, "no" #ifdefs -- or conceptually each file can be viewed as wrapped in an #ifdef or other platform detection.You need #ifdef in any C that uses 64 bit integers, for example. Current Microsoft compilers do support "long long" but __int64 supporg goes way back, and __int64 is a better name anyway. What is the 128 bit integer type to be called -- "__int128" or "long long long". The names "char", "short", "int", "long", "long long" are mostly just funny mistaken names. The right names to provide are explicitly sized, int8, uint8, int16, uint16, int32, uint32, int64, uint64, and pointer sized types such as "size_t" and "diff_t". Modula-3 I think is reasonable here, since it's primary integral type is pointer sized, and it has "bits x for ...", though the other types probably ought to be built in and blessed somewhere, not just in the Csupport library section. Not sure what "Posix" means, since it might incorporate by-reference ANSI C. - Jay Date: Sat, 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3devel at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Dear all:I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have.However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not claim any more they are safe so that "fix" is not feasible.Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file.Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ), that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation.There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that?Thanks in advance--- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is ok now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Dec 22 15:20:12 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 09:20:12 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229855598.18256.168.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> <1229855598.18256.168.camel@faramir.m3w.org> Message-ID: <20081222142011.GB28224@topoi.pooq.com> On Sun, Dec 21, 2008 at 11:33:18AM +0100, Dragi?a Duri? wrote: > Most important argument for UTF-8 - IMO - would be it's completeness. > 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some > obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit > storage magic instead? Well, 20 bits would certianly fit inside 16 bytes, :-) -- hendrik From darko at darko.org Sun Dec 21 22:07:21 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 06:07:21 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> The way to handle this would be to require an iterate function on each text representation which successively returned the next character in a string as a UNICHAR with defined characteristics. That would allow for comparison and transfer of strings. The way I see it, TEXT should be revealed to be a subtype of the "TextObj" object that has methods that basically mirror the Text interface with some additions. The existing text representation code is adapted to this object (fairly simple) and new representations subclass that object. A new function is added to text interface that allows the user to specify the default representation used when creating new strings. This would be the typecode of the object to allocate, required to be a subtype of TextObj of course. Some TextObj methods would be required to be implemented by the representation but most could be handled by the ancestor. Many of the methods could be implemented by the encoding for more efficient handling, for instance when the text being compared or concatenated is the same representation. UNICHAR = ; TextObj = OBJECT METHODS length(t: T): CARDINAL; (* required to be implemented by the encoding *) empty(t: T): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(t: T): BOOLEAN; (* required - meaning containing anything other than CHAR values *) next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index otherwise meaningless and private. Seek allows skipping forward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(t, u: T): BOOLEAN; compare(t1, t2: T): [-1..1]; cat(t, u: T): T; sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(t: T): Word.T; getChar(t: T; i: CARDINAL): CHAR; getWideChar(t: T; i: CARDINAL): WIDECHAR; getUniChar(t: T; i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWIdeChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(t: T; c: CHAR; start := 0): INTEGER; findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; Additionally the Text and a couple of other interfaces (eg Rd, Wr) would need to be expanded to handle UniChar. On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >> The right way to do this, IMNSHO is to not assume any particular >> representation of TEXT values and create an implementation interface >> that allows implementations of multiple text representations, much >> like >> Rd and Wr don't make many assumptions about how data is actually >> stored >> or retrieved. > > Such an interface may be needed for UTF-8 alone already, anyway, > because within UTF-8 there is in some cases more than one way > to store what amounts to the same data to a human user. > > In Subversion, from the beginning everyone agreed that the internal > encoding for all strings would be UTF-8. Most Subversion APIs expect > data in UTF-8. Strings (e.g. filenames) in the repository are stored > in UTF-8, etc. Great! Will work in all countries! Right? > > Yes, but not on all operating systems if you're not careful! > It did not occur to anyone at the time that there are characters > which in UTF-8 have more than one representation (codepoints) in a > byte stream. For example, an u with umlaut can be encoded as two > bytes or a single byte: > > 2 bytes: [u | the previous character has an umlaut ] > This is called "normal form decomposed". > > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) > This is called "normal form composed". > > If you want to be portable, as CM3 and Subversion both try to be, > you have to consider that some operating systems may return your > filenames in a different encoding then you stored it in: > > -------- > Accepts Gives back > MacOS X * NFD(*) > Linux * > Windows * > Others ? ? > > > *) There are some remarks to be made regarding full or partial > NFD here, but the essential thing is: If you send in NFC, don't > expect it back! > -------- quoted from: > http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames > which is worth a read for more details if you're interested. > > In Subversion, this is a real problem for Mac users, because > two filenames which only differ in their NFC/NFD encoding > look exactly the same to the user (an u umlaut is printed), > while the byte streams do not match ("We're sorry, but your > file x does not exist in the repository!", where x looks just > like a file that is clearly visible in the repository listing :) > > Subversion's problem now is that there are repositories out > there using filenames in either NFC, NFD, or mixed, and there > is no good way to reconcile the mess while staying backwards > compatible with existing clients, servers, working copies and > repositories. So Mac users are told to only use ASCII characters > in their filenames to prevent the problem (many users, especially > users who are not programmers, who store their photos or their > entire home directory or whatever in Subversion, are not happy > about this). > > This problem may not matter as much in case of CM3, but anyone > implementing UTF-8 support for CM3 should be aware of this issue > and not repeat the mistake the Subversion developers made at the > time! With UTF-8, do not rely on a filename to retain its encoding > as you passed it to the OS when requesting the filename from the > OS again. > > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for > filenames only, or for all strings, whichever makes more sense. > And then stick to it, converting input/output as needed. > > Abstracting this problem away using a nice interface would probably > be the cleanest solution. > > Stefan From hendrik at topoi.pooq.com Mon Dec 22 16:54:38 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 10:54:38 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <20081222155438.GA28375@topoi.pooq.com> On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: > The way to handle this would be to require an iterate function on each > text representation which successively returned the next character in > a string as a UNICHAR with defined characteristics. That would allow > for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some TextObj > methods would be required to be implemented by the representation but > most could be handled by the ancestor. Many of the methods could be > implemented by the encoding for more efficient handling, for instance > when the text being compared or concatenated is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) How is T related to TextObj? Does this return the number of characters of the number of bytes occupied by the stinrg? > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) Does this require a scan of the string to determine whether any wide characters are actually present, or does it just indicate whether the data representation used in this string is capable of handling wide characters? > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the next > logical character in char and returns true, or false if the char > doesn't exist. Index otherwise meaningless and private. Seek allows > skipping forward a number of characters. *) Do we need a colon after the second VAR? This seems intended for an implementation where index is a byte offset. Is it possible to copy index so as to start a new interatin where at a saved point? > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) What stack? > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) length in bytes or characters? > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; I'd really like functions that provide access to the underlying UTF8 encoding on a byte-by-byte basis. Often that's the most efficient way, and the simplest, to process UTF8-encoded data, and UTF8 had been designed with this in mind. > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > while we are at it, let's include byte access as well. Treating bytes as integers between 0 and 255 would work. -- hendrik Note: when I'm processing UTF8 data, I'm often parsing it and inserting other tokens (which are not UTF8 characters) into the data stream as well. It is convenient to use negative numbers for this, clearly distinguished from Unicode codepints, which are positive. So making characters availabla as positive integers would work cleanly with this. From darko at darko.org Mon Dec 22 01:07:39 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:07:39 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081222155438.GA28375@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> Message-ID: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> On 23/12/2008, at 12:54 AM, hendrik at topoi.pooq.com wrote: > On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: >> The way to handle this would be to require an iterate function on >> each >> text representation which successively returned the next character in >> a string as a UNICHAR with defined characteristics. That would allow >> for comparison and transfer of strings. >> >> The way I see it, TEXT should be revealed to be a subtype of the >> "TextObj" object that has methods that basically mirror the Text >> interface with some additions. The existing text representation code >> is adapted to this object (fairly simple) and new representations >> subclass that object. A new function is added to text interface that >> allows the user to specify the default representation used when >> creating new strings. This would be the typecode of the object to >> allocate, required to be a subtype of TextObj of course. Some TextObj >> methods would be required to be implemented by the representation but >> most could be handled by the ancestor. Many of the methods could be >> implemented by the encoding for more efficient handling, for instance >> when the text being compared or concatenated is the same >> representation. >> >> UNICHAR = ; >> >> TextObj = OBJECT METHODS >> >> length(t: T): CARDINAL; (* required to be implemented by the >> encoding *) > > How is T related to TextObj? Does this return the number of > characters > of the number of bytes occupied by the stinrg? Sorry, T = TEXT. It returns the logical number of characters according to some definition, which I imagine would be Unicode NFC (Normalised Form, Canonical Composition). > > >> empty(t: T): BOOLEAN; (* required to be implemented by the encoding >> *) >> hasWideChars(t: T): BOOLEAN; (* required - meaning containing >> anything other than CHAR values *) > > Does this require a scan of the string to determine whether any wide > characters are actually present, or does it just indicate whether the > data representation used in this string is capable of handling wide > characters? No to both. It doesn't require a scan, you can cache this flag. It's so you can possibly be more efficient when you know that you only have to deal with CHAR values. > > >> next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): >> BOOLEAN; (* required - start iterating with index=0, returns the next >> logical character in char and returns true, or false if the char >> doesn't exist. Index otherwise meaningless and private. Seek allows >> skipping forward a number of characters. *) > > Do we need a colon after the second VAR? No, it's a typo. > This seems intended for an implementation where index is a byte > offset. > Is it possible to copy index so as to start a new interatin where at a > saved point? Yes. > > >> getData(): ADDRESS; (* required - get the raw data, only valid >> while on the stack *) > > What stack? The execution stack. It might be a REFANY so would change if it didn't appear in the stack. > > >> setData(adr: ADDRESS; length: CARDINAL); (* required - set the data >> and length of the encoded data *) > > length in bytes or characters? Bytes, it's the length of the raw data and has no logical interpretation. > > >> >> (* the remaining methods are optional for the encoding >> implementation *) >> equal(t, u: T): BOOLEAN; >> compare(t1, t2: T): [-1..1]; >> cat(t, u: T): T; >> sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; >> hash(t: T): Word.T; >> getChar(t: T; i: CARDINAL): CHAR; >> getWideChar(t: T; i: CARDINAL): WIDECHAR; >> getUniChar(t: T; i: CARDINAL): UNICHAR; > > I'd really like functions that provide access to the underlying UTF8 > encoding on a byte-by-byte basis. Often that's the most efficient > way, > and the simplest, to process UTF8-encoded data, and UTF8 had been > designed with this in mind. If you know the text is encoded as UTF-8 then you can get the raw data as above. > > >> setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); >> setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); >> setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); >> fromChar(ch: CHAR): T; >> fromWIdeChar(ch: WIDECHAR): T; >> fromUniChar(ch: UNICHAR): T; >> fromChars(READONLY a: ARRAY OF CHAR): T; >> fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; >> fromUniChars(READONLY a: ARRAY OF UNICHAR): T; >> findChar(t: T; c: CHAR; start := 0): INTEGER; >> findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; >> findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; >> findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; >> findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; >> findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; >> END; >> >> Additionally the Text and a couple of other interfaces (eg Rd, Wr) >> would need to be expanded to handle UniChar. >> > > while we are at it, let's include byte access as well. Treating bytes > as integers between 0 and 255 would work. Can't see the point since you can ORD a char value to get the same. > > > -- hendrik > > Note: when I'm processing UTF8 data, I'm often parsing it and > inserting > other tokens (which are not UTF8 characters) into the data stream as > well. It is convenient to use negative numbers for this, clearly > distinguished from Unicode codepints, which are positive. So making > characters availabla as positive integers would work cleanly with > this. > > From darko at darko.org Mon Dec 22 01:14:16 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:14:16 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> A couple of corrections: - T = TEXT and TextObj <: T so it should read TextObj = T OBJECT - the cat method is required for the encoding implementation (helps to actually be able to create strings) - the CARDINAL in the next method should be INTEGER to enable searching backwards (findCharR) and should probably be named 'iter'. On 22/12/2008, at 6:07 AM, Darko wrote: > The way to handle this would be to require an iterate function on > each text representation which successively returned the next > character in a string as a UNICHAR with defined characteristics. > That would allow for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some > TextObj methods would be required to be implemented by the > representation but most could be handled by the ancestor. Many of > the methods could be implemented by the encoding for more efficient > handling, for instance when the text being compared or concatenated > is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the > next logical character in char and returns true, or false if the > char doesn't exist. Index otherwise meaningless and private. Seek > allows skipping forward a number of characters. *) > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > > > > On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > >> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >>> The right way to do this, IMNSHO is to not assume any particular >>> representation of TEXT values and create an implementation interface >>> that allows implementations of multiple text representations, much >>> like >>> Rd and Wr don't make many assumptions about how data is actually >>> stored >>> or retrieved. >> >> Such an interface may be needed for UTF-8 alone already, anyway, >> because within UTF-8 there is in some cases more than one way >> to store what amounts to the same data to a human user. >> >> In Subversion, from the beginning everyone agreed that the internal >> encoding for all strings would be UTF-8. Most Subversion APIs expect >> data in UTF-8. Strings (e.g. filenames) in the repository are stored >> in UTF-8, etc. Great! Will work in all countries! Right? >> >> Yes, but not on all operating systems if you're not careful! >> It did not occur to anyone at the time that there are characters >> which in UTF-8 have more than one representation (codepoints) in a >> byte stream. For example, an u with umlaut can be encoded as two >> bytes or a single byte: >> >> 2 bytes: [u | the previous character has an umlaut ] >> This is called "normal form decomposed". >> >> 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) >> This is called "normal form composed". >> >> If you want to be portable, as CM3 and Subversion both try to be, >> you have to consider that some operating systems may return your >> filenames in a different encoding then you stored it in: >> >> -------- >> Accepts Gives back >> MacOS X * NFD(*) >> Linux * >> Windows * >> Others ? ? >> >> >> *) There are some remarks to be made regarding full or partial >> NFD here, but the essential thing is: If you send in NFC, don't >> expect it back! >> -------- quoted from: >> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames >> which is worth a read for more details if you're interested. >> >> In Subversion, this is a real problem for Mac users, because >> two filenames which only differ in their NFC/NFD encoding >> look exactly the same to the user (an u umlaut is printed), >> while the byte streams do not match ("We're sorry, but your >> file x does not exist in the repository!", where x looks just >> like a file that is clearly visible in the repository listing :) >> >> Subversion's problem now is that there are repositories out >> there using filenames in either NFC, NFD, or mixed, and there >> is no good way to reconcile the mess while staying backwards >> compatible with existing clients, servers, working copies and >> repositories. So Mac users are told to only use ASCII characters >> in their filenames to prevent the problem (many users, especially >> users who are not programmers, who store their photos or their >> entire home directory or whatever in Subversion, are not happy >> about this). >> >> This problem may not matter as much in case of CM3, but anyone >> implementing UTF-8 support for CM3 should be aware of this issue >> and not repeat the mistake the Subversion developers made at the >> time! With UTF-8, do not rely on a filename to retain its encoding >> as you passed it to the OS when requesting the filename from the >> OS again. >> >> CM3 should pick either NFD or NFC as internal UTF-8 encoding, for >> filenames only, or for all strings, whichever makes more sense. >> And then stick to it, converting input/output as needed. >> >> Abstracting this problem away using a nice interface would probably >> be the cleanest solution. >> >> Stefan > From hendrik at topoi.pooq.com Mon Dec 22 21:10:36 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 15:10:36 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> Message-ID: <20081222201036.GC28728@topoi.pooq.com> On Mon, Dec 22, 2008 at 09:07:39AM +0900, Darko wrote: > > > > > >> getData(): ADDRESS; (* required - get the raw data, only valid > >>while on the stack *) > > > >What stack? > > The execution stack. It might be a REFANY so would change if it didn't > appear in the stack. I must be misunderstanding something; this seems perfectly obvious to you, but opaque to me. > >> > > > >while we are at it, let's include byte access as well. Treating bytes > >as integers between 0 and 255 would work. > > Can't see the point since you can ORD a char value to get the same. Not unless it's *specified* that the raw data structure *is* a bunch of CHARs, and that CHARs are single bytes. This whole discussion is occasioned by the discovery that characters are no longer single bytes, but strings of bytes. -- hendrik From darko at darko.org Mon Dec 22 03:52:38 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 11:52:38 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> Message-ID: <5A505AA9-97B5-4B20-878C-0FDC5359B8A7@darko.org> Ah the perils of copy and paste early in the morning. Here is a corrected version without extraneous method parameters. UNICHAR = ; T = TEXT; T <: TextObj; TextObj = OBJECT METHODS length(): CARDINAL; (* required to be implemented by the encoding *) empty(): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(): BOOLEAN; (* required - meaning containing anything other than CHAR values *) cat(u: T): T; (* required to be implemented by the encoding *) iter(VAR index: INTEGER; VAR char: UNICHAR; seek: INTEGER := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index is otherwise meaningless and private. Seek allows skipping forward or backward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(u: T): BOOLEAN; compare(u: T): [-1..1]; sub(start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(): Word.T; getChar(i: CARDINAL): CHAR; getWideChar(i: CARDINAL): WIDECHAR; getUniChar(i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWideChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWideChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(c: CHAR; start := 0): INTEGER; findWideChar(c: WIDECHAR; start := 0): INTEGER; findUniChar(c: UNICHAR; start := 0): INTEGER; findCharR(c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; PROCEDURE SetTextEnc(tc: INTEGER); (* set the encoding object to use from its typecode, must be subtype of TextObj *) PROCEDURE GetTextEnc(): INTEGER: (* returns the typecode of the current encoding object in use *) From hosking at cs.purdue.edu Mon Dec 22 21:56:50 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 23 Dec 2008 07:56:50 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: > From: Tony Hosking > Date: 23 December 2008 00:04:20 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:30:06 -- checkout in progress. > [start checkout 2008.12.22 06:30:10] > cd /tmp/build-cm3-20081222-063006-5jaOwe/build > cvs return value: 0 > [end checkout 2008.12.22 06:49:53] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 06:49:55 -- compile in progress. > [start compile 2008.12.22 06:49:55] > compile return value: 0 > [end compile 2008.12.22 06:55:09] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:57:16 -- checkout in progress. > [start checkout 2008.12.22 06:57:18] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > cvs return value: 0 > [end checkout 2008.12.22 07:16:39] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 07:16:41 -- compile in progress. > [start compile 2008.12.22 07:16:41] > compile return value: 0 > [end compile 2008.12.22 08:02:05] > COMPILE_RETURN = 0 > 2008.12.22 08:02:22 -- tests in progress. > [start run-tests 2008.12.22 08:02:22] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > [end run-tests 2008.12.22 08:02:22] > TESTS_RETURN = 0 > 2008.12.22 08:02:22 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 04:06:37 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Mon, 22 Dec 2008 21:06:37 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> References: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Message-ID: <495055BD.2010009@wichita.edu> Mika Nystrom wrote: > Note to self and others: m3gdb is not completely useful here. If > you look at the structure of a TextCat.T, you see that fields a and > b are TEXTs, but m3gdb doesn't really let you look inside them, > since, well, they are TEXTs and printed as such. (You don't > get to see the whole gory mess of pointers.) Is there a way to > turn off this (normally very helpful) m3gdb feature? Yes. Perhaps a couple of years ago, I added a /k format letter to m3gdb for printing TEXT values showing the internal object, rather than the characters in quotes. At the time, I only announced this in the CVS checkin logs (which show up in the m3commit mailing list), but recently, I added a good-sized bit of collected documentation for m3gdb, which describes this in more detail. From: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html#id2517604 ----------------------------------------------------------------------------- Normally, m3gdb displays TEXT values in Modula-3 lexical syntax, with double quotes, escape sequences, and, if appropriate, a leading 'W'. However, the /k option in a print command will cause it instead to display the TEXT value's internal data structure. For PM3 et. al., this is a traced reference to an open array of characters. For CM3, this will be one of the several object subtypes of TEXT. m3gdb properly recognizes and displays values of CM3 type TextLiteral.T. Here, it takes the length of the text from the appropriate field, instead of from the declared type, which contains a fixed array whose length is amost always far too long. Even without specifying the /k option, if you happen to know what the internal representation is, you can apply appropriate operators to it, e.g., dereferencing, field selection, subscripting, or even method calls. ----------------------------------------------------------------------------- Examples: (m3gdb) p T6 $15 = (*16_b740e1dc*) "TextLiteralabcdefghij" (m3gdb) p/k T6 $16 = (*16_b740e1dc*) OBJECT a = 16_0804a670; b = 16_b740e0c8; a_len = 11; b_len = 10; a_or_b_wide = FALSE; END (m3gdb) p/k T6.a $17 = (*16_0804a670*) (*TextLiteral.T*) cnt = 11; buf = "TextLiteral" END (m3gdb) p/k T6.b $18 = (*16_b740e0c8*) OBJECT len = 10; contents = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', '\000', '\000', '\000', '\000', '\000', '\000'}; END ----------------------------------------------------------------------------- It would be nicer if m3gdb also always printed the name of the type of the object, but I haven't figured out what to do here in general, because an object type can have no name, or more than one name. But I see I am now printing (*TextLiteral.T*) as a special case, so the other CM3 subtypes of TEXT could be handled the same way. 'Nother item for my long m3gdb to-do list. Rodney Bates > > Mika > > From jay.krell at cornell.edu Tue Dec 23 13:53:26 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 12:53:26 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Drat. I fixed a likely problem (broken on other platforms) and I think "upgrade" succeeded, but then I messed things up -- no working gcc. It looks like I might not be home now till just after New Years due to unforseen circumstances. (very inclement weather, stuck elsewhere) Can you get a more specific error message? If so, I can checkin a guess late in the week. Otherwise, can it wait? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Tue, 23 Dec 2008 07:56:50 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: From: Tony Hosking Date: 23 December 2008 00:04:20 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:30:06 -- checkout in progress.[start checkout 2008.12.22 06:30:10]cd /tmp/build-cm3-20081222-063006-5jaOwe/buildcvs return value: 0[end checkout 2008.12.22 06:49:53]CHECKOUT_RETURN = 0--2008.12.22 06:49:55 -- compile in progress.[start compile 2008.12.22 06:49:55]compile return value: 0[end compile 2008.12.22 06:55:09]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081222-063006-5jaOwe ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:57:16 -- checkout in progress.[start checkout 2008.12.22 06:57:18]cd /tmp/build-cm3-20081222-065716-Oha4Uk/buildcvs return value: 0[end checkout 2008.12.22 07:16:39]CHECKOUT_RETURN = 0--2008.12.22 07:16:41 -- compile in progress.[start compile 2008.12.22 07:16:41]compile return value: 0[end compile 2008.12.22 08:02:05]COMPILE_RETURN = 02008.12.22 08:02:22 -- tests in progress.[start run-tests 2008.12.22 08:02:22]cd /tmp/build-cm3-20081222-065716-Oha4Uk/build[end run-tests 2008.12.22 08:02:22]TESTS_RETURN = 02008.12.22 08:02:22 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Dec 23 13:59:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 23 Dec 2008 13:59:00 +0100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Quoting Tony Hosking : > Jay, things are still failing on my SOLgnu installation after your > changes. Did you ever try it on yours? According to tinderbox, all release builds fail due to undefined quake procedures: /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? " m3cc 1196 === package m3-sys/m3cc === 1197 +++ cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? +++ 1198 --- building in SOLgnu --- 1199 1200 NEXT "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: fs_exists 1201 fs_exists cannot be used unless we provide an official release before, because it has not been officially released yet. Some lastok-builds fail for other reasons. Has anybody time to fix these? Alternatively, we could back out the offending changes... > Happy holidays to all! Happy Christmas to all M3 users from me, too! Olaf > Begin forwarded message: > >> From: Tony Hosking >> Date: 23 December 2008 00:04:20 GMT+11:00 >> To: hosking at cs.purdue.edu >> Subject: Output from "cron" command >> >> Your "cron" job on niagara.cs.purdue.edu >> $HOME/cm3/scripts/regression/cron.sh >> >> produced the following output: >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara release-build" >> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:30:06 -- checkout in progress. >> [start checkout 2008.12.22 06:30:10] >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build >> cvs return value: 0 >> [end checkout 2008.12.22 06:49:53] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 06:49:55 -- compile in progress. >> [start compile 2008.12.22 06:49:55] >> compile return value: 0 >> [end compile 2008.12.22 06:55:09] >> COMPILE_RETURN = 1 >> *** COMPILE FAILED >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" >> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:57:16 -- checkout in progress. >> [start checkout 2008.12.22 06:57:18] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> cvs return value: 0 >> [end checkout 2008.12.22 07:16:39] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 07:16:41 -- compile in progress. >> [start compile 2008.12.22 07:16:41] >> compile return value: 0 >> [end compile 2008.12.22 08:02:05] >> COMPILE_RETURN = 0 >> 2008.12.22 08:02:22 -- tests in progress. >> [start run-tests 2008.12.22 08:02:22] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> [end run-tests 2008.12.22 08:02:22] >> TESTS_RETURN = 0 >> 2008.12.22 08:02:22 -- checkout, compile and test run done. >> >> --- >> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> done. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 23 14:07:20 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 13:07:20 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Message-ID: ah. I thought for some reason that m3cc could use newer stuff, probably confusing with m3tests, and I wouldn't catch this as I back up to older versions rarely. That's /trivial/ to fix..though I'm still hampered by unusual real-world circumstance, should get relief by end of week (Friday), or maybe much earlier, but certainly during Friday day. The fix is to just provide one's own, like: if not defined("fs_exists") proc fs_exists(a) is return state(a, a) end end Thanks, - Jay> Date: Tue, 23 Dec 2008 13:59:00 +0100> From: wagner at elegosoft.com> To: m3devel at elegosoft.com> Subject: Re: [M3devel] Fwd: Output from "cron" command> > Quoting Tony Hosking :> > > Jay, things are still failing on my SOLgnu installation after your> > changes. Did you ever try it on yours?> > According to tinderbox, all release builds fail due to undefined> quake procedures:> > > /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh > -c "cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? " m3cc> 1196 === package m3-sys/m3cc ===> 1197 +++ cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? +++> 1198 --- building in SOLgnu ---> 1199> 1200 NEXT > "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: > fs_exists> 1201> > fs_exists cannot be used unless we provide an official release before,> because it has not been officially released yet.> > Some lastok-builds fail for other reasons.> > Has anybody time to fix these? Alternatively, we could back out the> offending changes...> > > Happy holidays to all!> > Happy Christmas to all M3 users from me, too!> > Olaf> > > Begin forwarded message:> >> >> From: Tony Hosking > >> Date: 23 December 2008 00:04:20 GMT+11:00> >> To: hosking at cs.purdue.edu> >> Subject: Output from "cron" command> >>> >> Your "cron" job on niagara.cs.purdue.edu> >> $HOME/cm3/scripts/regression/cron.sh> >>> >> produced the following output:> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara release-build"> >>> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:30:06 -- checkout in progress.> >> [start checkout 2008.12.22 06:30:10]> >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build> >> cvs return value: 0> >> [end checkout 2008.12.22 06:49:53]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 06:49:55 -- compile in progress.> >> [start compile 2008.12.22 06:49:55]> >> compile return value: 0> >> [end compile 2008.12.22 06:55:09]> >> COMPILE_RETURN = 1> >> *** COMPILE FAILED> >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"> >>> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:57:16 -- checkout in progress.> >> [start checkout 2008.12.22 06:57:18]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> cvs return value: 0> >> [end checkout 2008.12.22 07:16:39]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 07:16:41 -- compile in progress.> >> [start compile 2008.12.22 07:16:41]> >> compile return value: 0> >> [end compile 2008.12.22 08:02:05]> >> COMPILE_RETURN = 0> >> 2008.12.22 08:02:22 -- tests in progress.> >> [start run-tests 2008.12.22 08:02:22]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> [end run-tests 2008.12.22 08:02:22]> >> TESTS_RETURN = 0> >> 2008.12.22 08:02:22 -- checkout, compile and test run done.> >>> >> ---> >>> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> done.> > > > -- > Olaf Wagner -- elego Software Solutions GmbH> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 18:28:36 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 11:28:36 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <49511FC4.2050903@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 2^16-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. I think we already have a reasonably designed abstraction for TEXT, a bit of it built in to the language and the rest in Text.i3. Only problem 1) affects the abstraction. Addressing only 1) for now: It ought to cause minimal grief to just change WIDECHAR so it has a big enough value range for all the Unicode values, probably 32 bits in today's world. Surely nobody has written any code that assumes BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly hard change to adapt old code to, since the static rules of the language would point directly to most places that would need to be fixed. That leaves the wide TEXT literals. It is easy to forget, and I have to keep reminding myself, but (assuming again that nobody has gotten their fingers improperly into the implementation pie) there is currently no such thing as a "WIDETEXT" type. Both kinds of literals are of type TEXT. They are just different lexical rules for specifying literal values of type TEXT. A bit like '16' and '16_10' are different ways of writing the same value, with the same type INTEGER. This differs from the CHAR and WIDECHAR literals, which really are of different types. The one change needed to the W"..." literals would be to allow escape sequences inside for giving characters numerically. Right now, the \x0123 form of escape requires exactly 4 hex digits. If we added a new alternative escape letter, in addition to the 'x', that required, say, exactly 8 hex digits, then these literals could express characters in the needed extra space, without affecting existing code. I suppose, for consistency and completeness, we should also add a new octal escape sequence that was long enough for the full new range. And, we would also need to allow the same new escape sequences in WIDECHAR literals. This would solve the lack of sufficient range problem and have either minimal or zero impact on existing code. Compiler and library changes would not be trivial, but quite reasonable. I believe problems 2) and 3) can be addressed solely by working on the TEXT implementation - Rodney Bates From dragisha at m3w.org Tue Dec 23 19:43:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 19:43:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <49511FC4.2050903@wichita.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> Message-ID: <1230057780.10385.23.camel@faramir.m3w.org> As my mother's tongue uses two alphabets for writing, Latin covered by ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph correspondence and three digraphs in Latin variant I think I have over-average experience with non-Latin1 alphabets, in various areas. If I have to express what we call widetext literal in my code, I will have to work with Unicode tables and pick character by character. Tedious! What I would do is - switch my keyboard to either Latin or Cyrillic mapping and - imagine that!!! - just type! Thus getting UTF-8 characters into my source. My example literal would be: CONST MyNameInCyrillic = "??????? ?????"; MyNameInLatin = "Dragi?a Duri?"; You can see or not these glyphs, depending on your MUA and to some degree on MTA's in transit. With all WIDE* talk it is what I am using. Me being example guy from non-Latin1 world. How many of you are non-Latin1 people and using 16bit "W literals" ? dd On Tue, 2008-12-23 at 11:28 -0600, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values that > exceed 2^16-1. > ... > I think we already have a reasonably designed abstraction for TEXT, > a bit of it built in to the language and the rest in Text.i3. Only > problem 1) affects the abstraction. Addressing only 1) for now: > > It ought to cause minimal grief to just change WIDECHAR so it has a > big enough value range for all the Unicode values, probably 32 bits > in today's world. Surely nobody has written any code that assumes > BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > hard change to adapt old code to, since the static rules of the > language would point directly to most places that would need to be > fixed. > > That leaves the wide TEXT literals. It is easy to forget, and I have > to keep reminding myself, but (assuming again that nobody has gotten > their fingers improperly into the implementation pie) there is > currently > no such thing as a "WIDETEXT" type. Both kinds of literals are of > type > TEXT. They are just different lexical rules for specifying literal > values of type TEXT. A bit like '16' and '16_10' are different ways > of writing the same value, with the same type INTEGER. This differs > from the CHAR and WIDECHAR literals, which really are of different > types. > > The one change needed to the W"..." literals would be to allow > escape sequences inside for giving characters numerically. Right > now, the \x0123 form of escape requires exactly 4 hex digits. If > we added a new alternative escape letter, in addition to the 'x', that > required, say, exactly 8 hex digits, then these literals could express > characters in the needed extra space, without affecting existing code. > I suppose, for consistency and completeness, we should also add a new > octal escape sequence that was long enough for the full new range. > > And, we would also need to allow the same new escape sequences in > WIDECHAR > literals. > ... > > - Rodney Bates > -- Dragi?a Duri? From hendrik at topoi.pooq.com Wed Dec 24 14:00:15 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:00:15 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <20081224130015.GA1001@topoi.pooq.com> On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. I see it. All we need to do to make this work is say that Modula-3 programs are encoded in Unicode. That and some work under the hood so make it work. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? Well, I'm from the Latin-1 world. I've had to debug a compiler that was mistreating Korean strings (this was before Unicode), and I've written a program that used UTF-8 hiragana. But I only visit the nonLatin-1 world. I don't live there. -- hendrik From hendrik at topoi.pooq.com Wed Dec 24 14:10:30 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:10:30 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224130015.GA1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> Message-ID: <20081224131030.GD1001@topoi.pooq.com> On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > CONST > > MyNameInCyrillic = "??????? ?????"; > > MyNameInLatin = "Dragi?a Duri?"; > > > > You can see or not these glyphs, depending on your MUA and to some > > degree on MTA's in transit. > > I see it. All we need to do to make this work is say that Modula-3 > programs are encoded in Unicode. That and some work under the hood so > make it work. Well, probably a *lot* of work. -- hendrik From dragisha at m3w.org Tue Dec 23 20:34:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 20:34:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224131030.GD1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> Message-ID: <1230060840.10385.30.camel@faramir.m3w.org> On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > CONST > > > MyNameInCyrillic = "??????? ?????"; > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > degree on MTA's in transit. > > > > I see it. All we need to do to make this work is say that Modula-3 > > programs are encoded in Unicode. That and some work under the hood so > > make it work. > > Well, probably a *lot* of work. Not at all. I am already using it. Just typing into. What does not work and I am not using it "length in glyphs". Not important for me, but not hard to implement. Either directly in Modula-3 or using external C lib. Search for UTF-8 pattern in UTF-8 text works - of course. Also - partitioning of TEXT's by *any real* criteria. By position of CHAR, or by position of any substring. > > -- hendrik > -- Dragi?a Duri? From mika at async.caltech.edu Tue Dec 23 22:15:39 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 13:15:39 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Tue, 23 Dec 2008 11:28:36 CST." <49511FC4.2050903@wichita.edu> Message-ID: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> "Rodney M. Bates" writes: ... >It ought to cause minimal grief to just change WIDECHAR so it has a >big enough value range for all the Unicode values, probably 32 bits >in today's world. Surely nobody has written any code that assumes >BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly ... Minor issue, I know, but... Pickles? Mika From rodney.bates at wichita.edu Tue Dec 23 23:31:54 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:31:54 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <495166DA.2090606@wichita.edu> Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? Yes, I have been thinking about this too, and some implementation work would be required here. It should not be too hard to use the existing mechanism that pickles/unpickles INTEGER fields of differing sizes on different architectures and do the same thing with different sizes of WIDECHAR. And apparently having had my fingers inside of pickles recently more than anybody else, I would expect to be the one to do it. It is likely that network objects containing such fields would want similar treatment. > > Mika From rodney.bates at wichita.edu Tue Dec 23 23:45:47 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:45:47 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <49516A1B.8050903@wichita.edu> They look very believable on my email client. Perhaps I glossed over this. Yes, if your keyboard/editor can be set to handle the glyphs you want directly, then you can always use them in literals, of either kind, instead of escape sequences. I am sure we need more implementation work here, but I think the TEXT abstraction is fine in this respect. We still need escapes too, as an alternative for when using a keyboard/display setting isn't convenient, and it is for this purpose I am suggesting this extension to the TEXT literals, as part of making Unicode support complete. Dragi?a Duri? wrote: > As my mother's tongue uses two alphabets for writing, Latin covered by > ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph > correspondence and three digraphs in Latin variant I think I have > over-average experience with non-Latin1 alphabets, in various areas. > > If I have to express what we call widetext literal in my code, I will > have to work with Unicode tables and pick character by character. > Tedious! > > What I would do is - switch my keyboard to either Latin or Cyrillic > mapping and - imagine that!!! - just type! Thus getting UTF-8 characters > into my source. My example literal would be: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? > - Rodney Bates From rodney.bates at wichita.edu Wed Dec 24 00:22:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 17:22:35 -0600 Subject: [M3devel] This disgusting TEXT business Message-ID: <495172BB.9070906@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 216-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. As for 2), the implementation could be greatly improved without changing the existing TEXT abstraction. It could even be improved a lot without changing the existing data structures, only the algorithms. Back in March, I conjectured about improving get_chars for a concatenation by recursing on only one side and iterating on the other, but I never did anything about it. Text.Cat could flatten concatenations that are short into one of the atomic representations. It could perhaps do a two-level rebalance of concatenation trees. All these would help at least somewhat with the performance consequences of building a string one character at a time by concatenation. And of course, ignoring the needs of those who have violated the TEXT abstraction, we could come up with entirely new internal representations. We could even just add some new ones and support mixtures of both the old and new ones. This would really just be an expansion of what we already have, which has several representations, with all the operations dynamically checking and adapting to them. On the other hand, I do know from hard experience, that the implementation size and complexity go up surprisingly as the number of alternative representations goes up. It's really a sort of cartesion product of operations, operands for each operation, and representations for each operand. Something that has not been mentioned is the _space_ overhead of the existing concatenation scheme. Proponents of the pure or nearly-pure OO language design philosophy (e.g. Smalltalk, Java) seem pretty oblivious to how much you lose with lots of separately-allocated heap objects connected by pointers. Aside from the extra pointers, there is allocator/GC overhead per object and fragmentation loss. Then there's loss of reference locality, leading to bigger working sets, which spills back into time overhead. And with heap-allocated open arrays, there are additional 1+NoOfDimensions words as well. So I am strongly in favor of cutting down on the number of separate objects, wherever reasonable. Rodney Bates From hendrik at topoi.pooq.com Wed Dec 24 19:00:01 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 13:00:01 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230060840.10385.30.camel@faramir.m3w.org> References: <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> <1230060840.10385.30.camel@faramir.m3w.org> Message-ID: <20081224180001.GA1449@topoi.pooq.com> On Tue, Dec 23, 2008 at 08:34:00PM +0100, Dragi?a Duri? wrote: > On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > > > CONST > > > > MyNameInCyrillic = "??????? ?????"; > > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > > degree on MTA's in transit. > > > > > > I see it. All we need to do to make this work is say that Modula-3 > > > programs are encoded in Unicode. That and some work under the hood so > > > make it work. > > > > Well, probably a *lot* of work. > > Not at all. I am already using it. Just typing into. > > What does not work and I am not using it "length in glyphs". Not > important for me, but not hard to implement. Either directly in Modula-3 > or using external C lib. > > Search for UTF-8 pattern in UTF-8 text works - of course. Also - > partitioning of TEXT's by *any real* criteria. By position of CHAR, or > by position of any substring. I see. You are just using the existing TEXT to store the bytes of UTF-8. That seems one of the simplest ways of accomodating UTF-8, in fact. No conversions in or out, just work in a pure UTF-8 environment. The "work" I talked about would be the effert necessary to implement all the existing methods that use character counts as character counts. Treating them as using byte counts is indeed simple. What remains is to build operations that deal in these UTF-8 TEXTs as being strings of UTF-8 characters, with operations like fetching and storing characters instead of bytes. Perhaps what is really needed is clarity in the specification, so we know which operations to use on bytes, and which on characters, and abolish the myth that character counts are an efficient way of accessing characters. In retrospect, maybe there should have been a type BYTE, a type CHARACTER, and a type TEXT, all conceptually separate. TEXT could be accessed by byte or character operations; and indexes into TEXT would be byte offsets. This lacks some generality that might have been needed in the days of machines with word-addressable 29-bit words, but those days seem to be past. What seems to block introducing this approach seems to be the prevalence of code that uses CHARACTER as a synonym for BYTE. Do we, for some kind of compatibility, have to use CHARACTER when we mean BYTE, and something else when we want to say CHARACTER? In the long run, will be be more inconvenienced by having to rewrite code that uses CHARACTER to represent characters, or code that uses CHARACTER to represent bytes? It is already possible to use 0..255 to represent bytes, isn't it? Or am I wrong in this? Might this mean four-byte integers that happen to have a restricted range of values? -- hendrik From darko at darko.org Wed Dec 24 01:20:27 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:20:27 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <495172BB.9070906@wichita.edu> References: <495172BB.9070906@wichita.edu> Message-ID: <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> I agree with your list of problems, but you don't need a cartesian product of operations for string operations between multiple representations. All you need is iteration through the characters of a string and concatenation with another string for each representation. The object interface I proposed is little more than a wrapper for the existing Text interface plus the iteration method. The getData and setData methods in that object provide an interface to external APIs. On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values > that > exceed 216-1. > > 2) The CM3 TEXT implementation has serious inefficiencies in at > least some > realistic cases. > > 3) We want some kind of compatibility with other software. > > As for 2), the implementation could be greatly improved without > changing > the existing TEXT abstraction. It could even be improved a lot > without > changing the existing data structures, only the algorithms. > > Back in March, I conjectured about improving get_chars for a > concatenation > by recursing on only one side and iterating on the other, but I > never did > anything about it. Text.Cat could flatten concatenations that are > short > into one of the atomic representations. It could perhaps do a two- > level > rebalance of concatenation trees. All these would help at least > somewhat > with the performance consequences of building a string one character > at > a time by concatenation. > > And of course, ignoring the needs of those who have violated the TEXT > abstraction, we could come up with entirely new internal > representations. > We could even just add some new ones and support mixtures of both > the old > and new ones. This would really just be an expansion of what we > already > have, which has several representations, with all the operations > dynamically > checking and adapting to them. > > On the other hand, I do know from hard experience, that the > implementation > size and complexity go up surprisingly as the number of alternative > representations goes up. It's really a sort of cartesion product of > operations, operands for each operation, and representations for each > operand. > > Something that has not been mentioned is the _space_ overhead of the > existing concatenation scheme. Proponents of the pure or nearly-pure > OO language design philosophy (e.g. Smalltalk, Java) seem pretty > oblivious to how much you lose with lots of separately-allocated > heap objects connected by pointers. > > Aside from the extra pointers, there is allocator/GC overhead > per object and fragmentation loss. Then there's loss of > reference locality, leading to bigger working sets, which spills > back into time overhead. And with heap-allocated open arrays, > there are additional 1+NoOfDimensions words as well. So I am > strongly in favor of cutting down on the number of separate objects, > wherever reasonable. > > Rodney Bates From darko at darko.org Wed Dec 24 01:27:20 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:27:20 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Besides breaking pickles (which isn't minor to those it affects) it also has the effect of doubling the size of String16 arrays and any wide strings or arrays. Why change the meaning of an existing type, possibly breaking unknown software when we can just define another type? It's not like another built-in type will cause anyone problems. On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? > > Mika From mika at async.caltech.edu Wed Dec 24 01:29:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:29:15 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:27:20 +0900." <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Message-ID: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Darko writes: >Besides breaking pickles (which isn't minor to those it affects) it >also has the effect of doubling the size of String16 arrays and any >wide strings or arrays. Why change the meaning of an existing type, >possibly breaking unknown software when we can just define another >type? It's not like another built-in type will cause anyone problems. stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > >> "Rodney M. Bates" writes: >> ... >>> It ought to cause minimal grief to just change WIDECHAR so it has a >>> big enough value range for all the Unicode values, probably 32 bits >>> in today's world. Surely nobody has written any code that assumes >>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >> ... >> >> Minor issue, I know, but... Pickles? >> >> Mika From mika at async.caltech.edu Wed Dec 24 01:31:50 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:31:50 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:20:27 +0900." <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> Message-ID: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Might I suggest an interface somewhat like the Rd/Wr interfaces? One could provide a buffer into which characters are read en masse instead of going through the method calls for each character.. Of course for equality this isn't necessarily more efficient since you often can do an early-out: although I believe that Text.Equal (when performance matters) usually does return TRUE.... I think it is important that basic TEXT operations be very, very fast. Many people care about that. Many of those that don't wouldn't be using a compiled language anyhow. Mika Darko writes: >I agree with your list of problems, but you don't need a cartesian >product of operations for string operations between multiple >representations. All you need is iteration through the characters of a >string and concatenation with another string for each representation. >The object interface I proposed is little more than a wrapper for the >existing Text interface plus the iteration method. The getData and >setData methods in that object provide an interface to external APIs. > > >On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > >> I hear three problems with CM3 TEXT: >> >> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >> that >> exceed 216-1. >> >> 2) The CM3 TEXT implementation has serious inefficiencies in at >> least some >> realistic cases. >> >> 3) We want some kind of compatibility with other software. >> >> As for 2), the implementation could be greatly improved without >> changing >> the existing TEXT abstraction. It could even be improved a lot >> without >> changing the existing data structures, only the algorithms. >> >> Back in March, I conjectured about improving get_chars for a >> concatenation >> by recursing on only one side and iterating on the other, but I >> never did >> anything about it. Text.Cat could flatten concatenations that are >> short >> into one of the atomic representations. It could perhaps do a two- >> level >> rebalance of concatenation trees. All these would help at least >> somewhat >> with the performance consequences of building a string one character >> at >> a time by concatenation. >> >> And of course, ignoring the needs of those who have violated the TEXT >> abstraction, we could come up with entirely new internal >> representations. >> We could even just add some new ones and support mixtures of both >> the old >> and new ones. This would really just be an expansion of what we >> already >> have, which has several representations, with all the operations >> dynamically >> checking and adapting to them. >> >> On the other hand, I do know from hard experience, that the >> implementation >> size and complexity go up surprisingly as the number of alternative >> representations goes up. It's really a sort of cartesion product of >> operations, operands for each operation, and representations for each >> operand. >> >> Something that has not been mentioned is the _space_ overhead of the >> existing concatenation scheme. Proponents of the pure or nearly-pure >> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >> oblivious to how much you lose with lots of separately-allocated >> heap objects connected by pointers. >> >> Aside from the extra pointers, there is allocator/GC overhead >> per object and fragmentation loss. Then there's loss of >> reference locality, leading to bigger working sets, which spills >> back into time overhead. And with heap-allocated open arrays, >> there are additional 1+NoOfDimensions words as well. So I am >> strongly in favor of cutting down on the number of separate objects, >> wherever reasonable. >> >> Rodney Bates From darko at darko.org Wed Dec 24 01:35:37 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:35:37 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> References: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Message-ID: <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Those don't involve data stored in files, can be fixed with a recompile and are only an issue if you reveal the object implementation of TEXT. On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > Darko writes: >> Besides breaking pickles (which isn't minor to those it affects) it >> also has the effect of doubling the size of String16 arrays and any >> wide strings or arrays. Why change the meaning of an existing type, >> possibly breaking unknown software when we can just define another >> type? It's not like another built-in type will cause anyone problems. > > stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >> >> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >> >>> "Rodney M. Bates" writes: >>> ... >>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>> big enough value range for all the Unicode values, probably 32 bits >>>> in today's world. Surely nobody has written any code that assumes >>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>> ... >>> >>> Minor issue, I know, but... Pickles? >>> >>> Mika From mika at async.caltech.edu Wed Dec 24 01:41:20 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:41:20 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:35:37 +0900." <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Message-ID: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Sorry, I think I misinterpreted what you meant by "built-in type". Darko writes: >Those don't involve data stored in files, can be fixed with a >recompile and are only an issue if you reveal the object >implementation of TEXT. > >On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > >> Darko writes: >>> Besides breaking pickles (which isn't minor to those it affects) it >>> also has the effect of doubling the size of String16 arrays and any >>> wide strings or arrays. Why change the meaning of an existing type, >>> possibly breaking unknown software when we can just define another >>> type? It's not like another built-in type will cause anyone problems. >> >> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >> >>> >>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>> >>>> "Rodney M. Bates" writes: >>>> ... >>>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>>> big enough value range for all the Unicode values, probably 32 bits >>>>> in today's world. Surely nobody has written any code that assumes >>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>>> ... >>>> >>>> Minor issue, I know, but... Pickles? >>>> >>>> Mika From darko at darko.org Wed Dec 24 01:47:36 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:47:36 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> References: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Message-ID: <064090A2-A17E-4CBF-9B7B-3A5B575C5706@darko.org> The proposed interface is like Rd/Wr in that it has a procedural interface on an opaque type that is revealed to be implemented as an object for each type of Rd/Wr. But the idea is to avoid the "one true" representation (which what the buffer is) because efficiency means different things to different people. If you do a lot of Unicode text and an UTF-32 representation is probably fastest. If you want to save space or you deal mostly with ASCII then UTF-8 is probably best. Going through each character is needed if you don't know the representation of the string you are dealing with. If you do know the format of the string you can do other operations. Since most applications will use one representation you will mostly be doing operations between strings of the same representations, which will be very fast, for instance you might call a local OS API to do the work. Having application defined representations means that you can adapt the string representation and operations to what you need to give you the performance you're after, or to choose one from an existing library that works best. On 24/12/2008, at 9:31 AM, Mika Nystrom wrote: > Might I suggest an interface somewhat like the Rd/Wr interfaces? > One could provide a buffer into which characters are read en masse > instead of going through the method calls for each character.. > > Of course for equality this isn't necessarily more efficient since > you often can do an early-out: although I believe that Text.Equal > (when performance matters) usually does return TRUE.... > > I think it is important that basic TEXT operations be very, very > fast. Many people care about that. Many of those that don't > wouldn't be using a compiled language anyhow. > > Mika > > Darko writes: >> I agree with your list of problems, but you don't need a cartesian >> product of operations for string operations between multiple >> representations. All you need is iteration through the characters >> of a >> string and concatenation with another string for each representation. >> The object interface I proposed is little more than a wrapper for the >> existing Text interface plus the iteration method. The getData and >> setData methods in that object provide an interface to external APIs. >> >> >> On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: >> >>> I hear three problems with CM3 TEXT: >>> >>> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >>> that >>> exceed 216-1. >>> >>> 2) The CM3 TEXT implementation has serious inefficiencies in at >>> least some >>> realistic cases. >>> >>> 3) We want some kind of compatibility with other software. >>> >>> As for 2), the implementation could be greatly improved without >>> changing >>> the existing TEXT abstraction. It could even be improved a lot >>> without >>> changing the existing data structures, only the algorithms. >>> >>> Back in March, I conjectured about improving get_chars for a >>> concatenation >>> by recursing on only one side and iterating on the other, but I >>> never did >>> anything about it. Text.Cat could flatten concatenations that are >>> short >>> into one of the atomic representations. It could perhaps do a two- >>> level >>> rebalance of concatenation trees. All these would help at least >>> somewhat >>> with the performance consequences of building a string one character >>> at >>> a time by concatenation. >>> >>> And of course, ignoring the needs of those who have violated the >>> TEXT >>> abstraction, we could come up with entirely new internal >>> representations. >>> We could even just add some new ones and support mixtures of both >>> the old >>> and new ones. This would really just be an expansion of what we >>> already >>> have, which has several representations, with all the operations >>> dynamically >>> checking and adapting to them. >>> >>> On the other hand, I do know from hard experience, that the >>> implementation >>> size and complexity go up surprisingly as the number of alternative >>> representations goes up. It's really a sort of cartesion product of >>> operations, operands for each operation, and representations for >>> each >>> operand. >>> >>> Something that has not been mentioned is the _space_ overhead of the >>> existing concatenation scheme. Proponents of the pure or nearly- >>> pure >>> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >>> oblivious to how much you lose with lots of separately-allocated >>> heap objects connected by pointers. >>> >>> Aside from the extra pointers, there is allocator/GC overhead >>> per object and fragmentation loss. Then there's loss of >>> reference locality, leading to bigger working sets, which spills >>> back into time overhead. And with heap-allocated open arrays, >>> there are additional 1+NoOfDimensions words as well. So I am >>> strongly in favor of cutting down on the number of separate objects, >>> wherever reasonable. >>> >>> Rodney Bates From darko at darko.org Wed Dec 24 01:50:53 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:50:53 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> References: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Message-ID: Sorry I thought you were referring to TEXT representations. We could define UNICHAR = INTEGER, an alias, which would avoid those changes. That might be required since the Unicode character set seems to change constantly and it might not work to define it as an enumeration. On 24/12/2008, at 9:41 AM, Mika Nystrom wrote: > > Sorry, I think I misinterpreted what you meant by "built-in type". > > Darko writes: >> Those don't involve data stored in files, can be fixed with a >> recompile and are only an issue if you reveal the object >> implementation of TEXT. >> >> On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: >> >>> Darko writes: >>>> Besides breaking pickles (which isn't minor to those it affects) it >>>> also has the effect of doubling the size of String16 arrays and any >>>> wide strings or arrays. Why change the meaning of an existing type, >>>> possibly breaking unknown software when we can just define another >>>> type? It's not like another built-in type will cause anyone >>>> problems. >>> >>> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >>> >>>> >>>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>>> >>>>> "Rodney M. Bates" writes: >>>>> ... >>>>>> It ought to cause minimal grief to just change WIDECHAR so it >>>>>> has a >>>>>> big enough value range for all the Unicode values, probably 32 >>>>>> bits >>>>>> in today's world. Surely nobody has written any code that >>>>>> assumes >>>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a >>>>>> terribly >>>>> ... >>>>> >>>>> Minor issue, I know, but... Pickles? >>>>> >>>>> Mika From jay.krell at cornell.edu Wed Dec 24 09:00:39 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 24 Dec 2008 08:00:39 +0000 Subject: [M3devel] a text proposal?? Message-ID: NOTE I am not a vehement advocate of the below. I'm at least "floating" a "thought balloon". Like, doesn't this sound fairly ok and reasonable? I propose the following "one" "parameterized" representation: a block of memory with a stored allocated size, probably in bytes, but could be divided by character size (i.e. "length in chars" vs. "length in bytes") a stored character size, in bytes, which shall be 1, 2, or 4 a stored length, probably in bytes, but could be divided by character size a stored maximum character value, maybe This is too expensive to maintain in general. It could be "advisory". However, it is also somewhat redundant with the character size. There shall be one implementation, though part of it shall be in a generic module, parameterized by character size. (yeah, yeah, parameterized by whatever Modula-3 allows) Appending a string with character size n with a string with character size m shall result in a string with character size max(n, m). String append shall always result in a "flat array". Never linked lists. Strings are never "encoded". Characters shall only ever be zero extended. Maybe there shall be functions to create strings from encoded strings. e.g. from utf8 and whatever is the name for 16 bit unicode that contain 20 bit surrogate pairs or whatever they are called. And to encode strings. Concatenating shall grow the allocated size by at least a constant factor greater than one. Possibly 2, possibly smaller. If that fails, fallback to allocating only what is needed. [This probably makes no sense, given text immutability and that concat produces new texts. But it might make sense if "text" is a value type.] Concatenation of n texts shall be a primitive, not just 2 texts. [It looks like this is already the case.] I think TEXTs are considered immutable. Over-allocating for later in-place concatenation doesn't, I believe, contract that. However perhaps I misspoke, and the allocated size is stored with the pointer, and the text contains a pointer to that, and a length. And maybe an offset. But "internal pointers" are ok and therefore an offset not needed, right? Cop out rationale: Anyone who needs O(1) concatenation of very long strings, is probably using the wrong type. Since texts are immutable, it may be viable, or even cheap, to allow a text to stick around with multiple representations, possibly using "weak refs". That is, string shall have probably a function or functions to get at the raw chars. Either the user has to specify a char size he wants, and get back null if the wrong size, or perhaps if user asks for larger than is needed, the text realizes that representation on-demand? And then it sticks around, in case another client soon thereafter asks for the same wider-than-necessary representation? The first order implementation can be simpler though. It can either return null for all but the current representation, or realize and return, but not hold onto, whatever representation is requested. Note that I'm not familiar with the code (yet) so this might be partly or total nonsense. I'm particular uncertain about the ability to double allocation size as individual chars are appended to a text. Issues of normalization remain punted. They already exist. Functions could be provided for convert to composed, decomposed, etc. - Jay From dragisha at m3w.org Thu Dec 25 01:00:05 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 25 Dec 2008 01:00:05 +0100 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 Message-ID: <1230163205.7131.1.camel@faramir.m3w.org> Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode (via UTF-8 as internal rep) then we must also extend CHAR so it can represent any single Unicode glyph ? in fact CHAR becomes 32bit value instead of current 8bit. If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we are on UNICHAR route ? as proposed by Darko (IIRC). But ? down that road we have variations in very traditional interface Text.i3 ? Text.GetChar, Text.SetChars, Text.FromChar, Text.FromChars must have these two variants ? somehow. I see no elegant way to handle that if we insist on BITSIZE(CHAR) = 8. UNICHAR route also contains other branches, most of them analog to current Text8/Text16 mess. UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte buffers, is one problem. Not too hard to spot and fix, though. Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that way. If similar need happens in future, ie some runtime level data communication, I think we can do it at ?connection? level. Some marshalling would most probably always take place ? so why not add TEXT I/O to list of tasks needed? -- Dragi?a Duri? From dabenavidesd at yahoo.es Thu Dec 25 01:12:07 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 25 Dec 2008 00:12:07 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Dear all, actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the? helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix INTERFACE). In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. You can read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess)? http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on? back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32..m3.html ). Happy holidays and hope this helps a bit more. --- El lun, 22/12/08, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Para: "Jay" CC: mika at camembert.async.caltech.edu Fecha: lunes, 22 diciembre, 2008 10:49 I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now
-- changing my "=
>ino_t" to
"m3_ino_t".
I'm still
checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 -
 Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinbishop at bellsouth.net Thu Dec 25 03:14:31 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Wed, 24 Dec 2008 20:14:31 -0600 Subject: [M3devel] Modula-3 subreddit Message-ID: <4952EC87.5050709@bellsouth.net> Just wanted to say that I made a Modula-3 subreddit on Reddit (a while back) if anyone is interested in joining (and hopefully submitting interesting links :P) http://www.reddit.com/r/modula3/ From mika at async.caltech.edu Fri Dec 26 09:56:24 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 26 Dec 2008 00:56:24 -0800 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 In-Reply-To: Your message of "Thu, 25 Dec 2008 01:00:05 +0100." <1230163205.7131.1.camel@faramir.m3w.org> Message-ID: <200812260856.mBQ8uOZ0065080@camembert.async.caltech.edu> I think you may find "SET OF CHAR" hidden all over the place. Also a lot of code may depend on the fact that CHAR and C's "char" types match on a given platform. This is your UNSAFE objection, but I think it goes further than just buffers. Mika =?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode >(via UTF-8 as internal rep) then we must also extend CHAR so it can >represent any single Unicode glyph ??? in fact CHAR becomes 32bit value >instead of current 8bit. > >If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we >are on UNICHAR route ??? as proposed by Darko (IIRC). But ??? down that road >we have variations in very traditional interface Text.i3 ??? Text.GetChar, >Text.SetChars, Text.FromChar, Text.FromChars must have these two >variants ??? somehow. I see no elegant way to handle that if we insist on >BITSIZE(CHAR) = 8. > >UNICHAR route also contains other branches, most of them analog to >current Text8/Text16 mess. > >UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte >buffers, is one problem. Not too hard to spot and fix, though. > >Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that >way. If similar need happens in future, ie some runtime level data >communication, I think we can do it at ???connection??? level. Some >marshalling would most probably always take place ??? so why not add TEXT >I/O to list of tasks needed? > >-- >Dragi??a Duri?? From martinbishop at bellsouth.net Sat Dec 27 03:39:08 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Fri, 26 Dec 2008 20:39:08 -0600 Subject: [M3devel] Who runs modula3.org? Message-ID: <4955954C.5050102@bellsouth.net> Who runs modula3.org, or at least has access? Quite a few of the links are dead. Some link to m3.org, others link to gatekeeper (which is gone). Most have alternative pages hosted elsewhere, but almost all are archived at archive.org, if needed. From dragisha at m3w.org Sat Dec 27 08:02:11 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 27 Dec 2008 08:02:11 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <4955954C.5050102@bellsouth.net> References: <4955954C.5050102@bellsouth.net> Message-ID: <1230361331.7131.6.camel@faramir.m3w.org> Public reply, so I can be corrected by someone knowing better. WWW is AFAIK run by Elego, but content is version controlled from CVS, so IMO it can be changed usual way - by editing and commiting. I don't know about permissions and update procedures for this, but you can browse src repo a bit and see for yourself. On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: > Who runs modula3.org, or at least has access? Quite a few of the links > are dead. Some link to m3.org, others link to gatekeeper (which is > gone). Most have alternative pages hosted elsewhere, but almost all are > archived at archive.org, if needed. -- Dragi?a Duri? From wagner at elegosoft.com Sat Dec 27 15:43:49 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 27 Dec 2008 15:43:49 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <1230361331.7131.6.camel@faramir.m3w.org> References: <4955954C.5050102@bellsouth.net> <1230361331.7131.6.camel@faramir.m3w.org> Message-ID: <20081227154349.5lyajia0w0k88wko@mail.elegosoft.com> Quoting Dragi?a Duri? : > Public reply, so I can be corrected by someone knowing better. > > WWW is AFAIK run by Elego, but content is version controlled from CVS, > so IMO it can be changed usual way - by editing and commiting. > > I don't know about permissions and update procedures for this, but you > can browse src repo a bit and see for yourself. > > On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: >> Who runs modula3.org, or at least has access? Quite a few of the links >> are dead. Some link to m3.org, others link to gatekeeper (which is >> gone). Most have alternative pages hosted elsewhere, but almost all are >> archived at archive.org, if needed. Elego took over the hosting of modula3.org from Igencorp (formerly Critical Mass) run by Farshad Nayeri one or two years ago, I think. The sources are checked in somewhere on birch.elego.de (though I don't know the exact path offhand (I'm not at home right now)) and can be edited and shipped by anyone with ssh access to birch. You can also send me diffs and I'll try to apply them, which may take some days of course. Any improvements and corrections are very welcome! Olaf > -- > Dragi?a Duri? > > -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Sat Dec 27 20:48:46 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 19:48:46 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 Message-ID: fyi: I've been looking at Utime.i3. One of the variations is whether or not struct_tm contains gmtoff and zone at the end. C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 are wrong on some systems. They unconditionally add the two fields, and read from memory that doesn't necessarily have them. It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). I'll double check Solaris and Irix soon or next week. I'll try to come up with a fix shortly. It will surely involve deleting those two files. And breaking any users of Utime.localtime and Utime.gmtime. - Jay From jay.krell at cornell.edu Sat Dec 27 21:12:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 20:12:50 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 In-Reply-To: References: Message-ID: Are localtime_r and gmtime_r really not widely available these days? I don't have a Darwin system handy this week. That is, we have DateBsd.m3, DateLinux.m3, DatePosix.m3. Excerpt: readonly _DateImpls = { "AMD64_DARWIN": "DateBsd", "AMD64_LINUX": "DateBsd", "FreeBSD*" : "DateBsd", "LINUXLIBC6" : "DateBsd", "MIPS64_OPENBSD" : "DateBsd", "NT386" : "DateLinux", "NT386GNU" : "DateLinux", "SOLgnu" : "DatePosix", "SOLsun" : "DatePosix", "SPARC" : "DateBsd", "*_LINUX" : "DateBsd", "SPARC64_SOLARIS" : "DatePosix", "*_DARWIN" : "DateBsd", "*_OPENBSD" : "DateBsd", } DateLinux lacks reentreent functions, and lacks gmtoff/zone in struct tm. It is only used by Cygwin. And Linux 1.x which I assume doesn't matter. This appears to be an oversight by me, the reentrant functions exist. DateBsd lacks reentrant functions, and has gmtoff/zone in struct tm. It is very widely used. DatePosix has reentrant functions, and lacks gmtoff/zone in struct tm. I rather suspect that everyone can just use DatePosix. ? Or at least that DateBsd can use reentrant functions. Eliminating unnecessary locks/globals always seems good. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sat, 27 Dec 2008 19:48:46 +0000 > Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 > > > fyi: > > > I've been looking at Utime.i3. > One of the variations is whether or not struct_tm contains gmtoff and zone at the end. > > > C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 > C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 > > > are wrong on some systems. > They unconditionally add the two fields, and read from memory that doesn't necessarily have them. > > > It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. > It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). > I'll double check Solaris and Irix soon or next week. > > > I'll try to come up with a fix shortly. > It will surely involve deleting those two files. > And breaking any users of Utime.localtime and Utime.gmtime. > > > - Jay From mika at async.caltech.edu Tue Dec 23 04:49:25 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 22 Dec 2008 19:49:25 -0800 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: Your message of "Sun, 21 Dec 2008 17:57:00 GMT." Message-ID: <200812230349.mBN3nPem008337@camembert.async.caltech.edu> I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now -- changing my "=
>ino_t" to
"m3_ino_t".
I'm still checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 - Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- From jay.krell at cornell.edu Sun Dec 28 01:19:24 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:19:24 +0000 Subject: [M3devel] PPC_LINUX Message-ID: I think PPC_LINUX has been broken for a few years, and not released since 5.2.6. It should be ok now and current 5.2.7 min and std distributions are at http://modula3.elegosoft.com/cm3/uploaded-archives We might want to fix user threads? - Jay From jay.krell at cornell.edu Sun Dec 28 01:22:20 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:22:20 +0000 Subject: [M3devel] port names? Message-ID: Again..what do people want for port names? MSDOS? Nice and short in the old style. I386_MSDOS? I think so. DJGPP? Nice and short in the old style. I386_DJGPP? Good because it identifies more -- the compiler, linker, C runtime. As opposed to say "C_WATCOM_DOS" or "I386_DOS_WATCOM" or such. DOS? Again, nice and short in the old style. I386_SOLARIS? Presumbly. X86_SOLARIS? AMD32_SOLARIS? :) IA32_SOLARIS? - Jay From jay.krell at cornell.edu Sun Dec 28 12:00:11 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 11:00:11 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <892265.61271.qm@web28603.mail.ukl.yahoo.com> References: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Message-ID: another take on this.. Cloning C headers in Modula-3 is both error prone and unsafe. It is not subject to static checking, is not verifiably correct, and errors lead to the same sort of type system violations that unsafe code can cause. I have found a few bugs here and there in the cloned headers, mostly wrt alignment. I replaced it with something probably still unsafe, but far less error prone. - Jay ________________________________ > Date: Thu, 25 Dec 2008 00:12:07 +0000 > From: dabenavidesd at yahoo.es > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > > Dear all, > actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. > I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix > INTERFACE). > In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most > operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). > Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. > Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. > You can > read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . > Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess) http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32.m3.html ). > > Happy holidays and hope this helps a bit more. > > > --- El lun, 22/12/08, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > Para: "Jay" > CC: mika at camembert.async.caltech.edu > Fecha: lunes, 22 diciembre, 2008 > 10:49 > > > I think Daniel has a point. He said "UNSAFE", not > "unsafe". "UNSAFE" > means something very specific in Modula-3, and operations that are > UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and > hence only can be used in UNSAFE MODULEs. > > If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that > is a problem that really shouldn't be allowed to get worse. > > This is all covered in sections 1.4.5 and 2.1 of the Green Book. > I'm not sure exactly what you guys are talking about but I would > appreciate it if I could rely on the Green Book for programming in > Modula-3.... > > Not everything that's is UNSAFE, by the way. It is > perfectly possible for a routine coded in C or Fortran to export a > non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can > definitely export an UNSAFE interface. > > I think if you interpret section 2.1 a bit loosely, > but > I think > still within the spirit of the authors' intent, you can go as far > as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE > interfaces) as long as the program needs to use at least one UNSAFE > INTERFACE in order to trigger the error. The contrapositive of > this is that it should not be possible for a Modula-3 program that > imports only non-UNSAFE interfaces to cause an unchecked runtime > error. Ever. > > Some examples...., in the following, assume that CrashHorribly > can cause an unchecked runtime error when called. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > This is OK since an importer has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT > A; > PROCEDURE CrashHorribly(arg : A.T); END C. > > Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > Might be OK, as long as the only way to get an unchecked runtime > error in C is by passing in a B.T. If you can get an unchecked > runtime error in C.CrashHorribly without importing B (say by passing > in a bare A.T or a "home-made" subtype of A), it's wrong. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C.. > > Definitely wrong. > > Mika > > > Jay writes: >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: > > quoted-printable >> >> >>The real world is unsafe.Safe areas are created by unsafe code=2C which > mus= >>t be=2C to a certain extent=2C bug free=2C in order to provide as safe as > a= >>dvertised. I don't believe what I added is any more untrusted/unsafe > than = >>what was there before.Well=2C maybe in a minor way. m3core has > "lots" of "= >>external" functions=2C which are called by "lots" of > functions at higher le= >>vels..ok=2C maybe not "lots"=2C but if you look at the > m3core/src/unix dire= >>ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C > linux-= >>common -- this is a fairly minimal set of stuff that is being used by > preex= >>isting code. A LITTLE bit I added beyond what was there. If you look at > all= >> the other directories=2C you'll find lots more external stuff=2C most > of w= >>hich is not used within the CM3 codebase (but could > be > used by folks > outsid= >>e). It is true I may have introduced external where it wasn't > previously= >>=2C but I don't think it matters.It is prevalent one level down. Moving > som= >>e of it one level up is ok I believe. =20 >>Non-Posix does potentially have a place in .c files.It > depends.Basically=2C= >> as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes > = >>hard to read=2C and at some point=2C you split the files into separate > file= >>s=2C "no" #ifdefs -- or conceptually each file can be viewed as > wrapped in = >>an #ifdef or other platform detection.You need #ifdef in any C that uses > 64= >> bit integers=2C for example. Current Microsoft compilers do support > "long = >>long" but __int64 supporg goes way back=2C and __int64 is a better > name any= >>way. >>What is the 128 bit integer type to be called -- "__int128" or > "long long > > l= >>ong". >>=20 >>=20 >>The names "char"=2C "short"=2C "int"=2C > "long"=2C "long long" are mostly ju= >>st funny mistaken names. >>The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >>=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > types= >> such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >>=20 >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >>=20 >> - Jay >> >> >> >>Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: > m3dev= >>el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin > was > Re: > [M3commit]= >> FilePosixC.c >> >> >> >>Dear all:I'm glad that Jay and all the people around are making > contributio= >>ns=2C so we can claim that as some day there were a good number of > supporte= >>d platforms nowadays we also have.However I have a question=2C but its > more= >> a suggestion=2C the modification Tony is referring is not as standard as > w= >>e would like since the type safety is affected with the change introduced > a= >>llowing (untrusted) functions in the INTERFACE > FilePosix=2C it= >>'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C > so = >>we may want to mark as UNSAFE INTERFACE=2C however doing that I think some > = >>clients could not claim any more they are safe so that "fix" is > not feasib= >>le.Can you push to a lower level (as in the UNSAFE FilePosix > implementation= >> module > before > committing the change) or make it available in other > Interfa= >>ce already available=2C why do we need that change anyway? Also IMHO we > sh= >>ould leave out what is not POSIX (we should not care for WIN64 although I > k= >>now is needed by the C compiler preprocessor) out of that file.Although I > d= >>on't have the formal specification or program=2C but I can remember > libm3 c= >>ode was specified and mechanically checked with the Extended Static > Checker= >> Modula-3 (which is not sound also=2C that is could be cheated=2C see > http:= >>//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 > )=2C = >> that would be a good reason for keeping as safe as possible and close to > h= >>istorical interfaces and just rework implementations (even if they are not > = >>safe) as needed. I know that implementation is not safe but at least > having= >> it well > enough > implemented the ESC/M3 checker and the compiler would tell > = >>us the library should work Ok and is safe and (hopefully) without the > error= >>s detected by the ESC as we care for the details in the UNSAFE > implementati= >>on.There are any special interfaces we should have as "standard" > in the low= >>er level runtime (hard enough question I know)=2C can we give some > hints/id= >>eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay >>>ll.edu> escribi=F3: >>De: Jay Asunto: Re: [M3devel] > serial/CygwinPara: "m3= >>devel" =2C "Tony" > Fecha: vier= >>nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ > it is o= >>k now -- changing my "ino_t" to"m3_ino_t".I'm still > checking though.And I o= >>nly checked SOLgnu. I will check SOLsun. - > Jay > = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>The real world is unsafe. Safe areas are created by unsafe code=2C > which= >> must be=2C to a certain extent=2C bug free=2C in order to provide as safe > = >>as advertised. =3B =3B I > don't believe what I added is a= >>ny more untrusted/unsafe than what was there before. Well=2C maybe > in a = >>minor way. =3B =3B m3core > has > "lots" of > "external" funct= >>ions=2C which are called by "lots" of functions at higher > levels..ok=2C may= >>be not "lots"=2C but if you look at the m3core/src/unix > directories that I = >>have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this > = >>is a fairly minimal set of stuff that is being used by preexisting code. A > = >>LITTLE bit I added beyond what was there. If you look at all the other > dire= >>ctories=2C you'll find lots more external stuff=2C most of which is not > use= >>d within the CM3 codebase (but could be used by folks > outside). =3B= >> =3B It is true I may have introduced external > where it wasn't p= >>reviously=2C but I don't think it matters. It is prevalent one > level dow= >>n. Moving some of it one level up is ok > I > believe. =3B =3B >> Non-Posix > does potentially have a place in .c files. It > depends. = >>Basically=2C as I see things=2C a small amount of #ifdef is ok. A > lot of= >> #ifdef becomes hard to read=2C and at some point=2C you split the files > in= >>to separate files=2C "no" #ifdefs -- or conceptually each file > can be viewe= >>d as wrapped in an #ifdef or other platform detection. You need > #ifdef i= >>n any =3BC that uses 64 bit integers=2C for example. Current > Microsoft&= >>nbsp=3Bcompilers do support "long long" but __int64 supporg goes > way back= >>=2C and __int64 is a better name anyway. >>What is the 128 bit integer type to be called -- "__int128" or > "long long l= >>ong". >> =3B >> =3B >>The names "char"=2C > "short"=2C "int"=2C > "long"=2C =3B"long long" = >>=3Bare mostly just > funny mistaken names. >>The right names to provide are explicitly sized=2C > int8=2C =3Buint8=2C = >>int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > = >>types such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >> =3B >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >> =3B >> =3B =3B- > Jay >> >> ________________________________ > id=3DstopSpelling> >> >> Date: Sat=2C 20 Dec 2008 00:09:41 > +0000 From: > dabenavidesd at yahoo.es<= >>BR>To: m3devel at elegosoft.com Subject: [M3devel] Rv: Re: > serial/Cygwin wa= >>s Re: [M3commit] FilePosixC.c >> >> >> >> Dear all: I'm glad that Jay and all the > people around a= >>re making contributions=2C so we can claim that as some day there were a > go= >>od number of supported platforms nowadays we also have. However I > have a= >> question=2C but its more a suggestion=2C the modification Tony is > referrin= >>g is not as standard as we would like since the type safety is affected > wit= >>h the change introduced allowing > <=3B> class=3DEC_current-rev>*>>>EXTERNAL> class=3DEC_current-rev>*>=3B > (untrusted) functions = >>in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 > (IMHO= >> it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C > = >>however doing that I think some clients could not =3B claim any > more th= >>ey are safe so that "fix" is not feasible. Can you push > to a lower level= >> (as in the UNSAFE FilePosix implementation module before committing the > ch= >>ange) or make it available in other Interface already available=2C why do > w= >>e need that change anyway? =3B Also IMHO we should leave out what > is no= >>t POSIX (we should not care for WIN64 although I know is needed by the C > co= >>mpiler preprocessor) out of that file. Although I don't have > the > formal = >>specification or program=2C but I can remember libm3 code was > specified > and= >> mechanically checked with the Extended Static Checker Modula-3 (which is > n= >>ot sound also=2C that is could be cheated=2C see > http://www.researchchannel= >>.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B > that wo= >>uld be a good reason for keeping as safe as possible and close to > historica= >>l interfaces and just rework implementations (even if they are not safe) > as= >> needed. I know that implementation is not safe but at least having it > well= >> enough implemented the ESC/M3 checker and the compiler would tell us the > l= >>ibrary should work Ok and is safe and (hopefully) without the errors > detect= >>ed by the ESC as we care for the details in the UNSAFE > implementation. T= >>here are any special interfaces we should have as "standard" > in > the lower l= >>evel runtime (hard enough question I know)=2C can we give > some hints/ideas > = >>on that? Thanks in > advance --- El vie=2C > 19/12/08=2C J= >>ay <=3Bjay.krell at cornell.edu>=3B > escribi=F3: >> > 5px">De: Jay <=3Bj= >>ay.krell at cornell.edu>=3B Asunto: Re: [M3devel] > serial/Cygwin Para: = >>"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C > "Tony" <=3Bhosking at cs.purd= >>ue.edu>=3B Fecha: viernes=2C 19 diciembre=2C 2008 > 1:25 >= >>=3B And I'll fix SOLsun/gnu. I /think/ it is ok now > -- changing my "= >>ino_t" to "m3_ino_t". I'm still > checking though. And I only > checke= >>d SOLgnu. I will check SOLsun. - > Jay > >>TD> >> = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- > > From martinbishop at bellsouth.net Sun Dec 28 23:14:14 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sun, 28 Dec 2008 16:14:14 -0600 Subject: [M3devel] Arbitrary precision integers Message-ID: <4957FA36.8080308@bellsouth.net> Does Modula-3 (or one of it's implementations) support arbitrary precision integers? From dabenavidesd at yahoo.es Sun Dec 28 23:48:35 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Dec 2008 22:48:35 +0000 (GMT) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> Message-ID: <460649.13613.qm@web28612.mail.ukl.yahoo.com> Hi, I guess BigInteger is a big integer of arbitrary size, you can look on http://www.opencm3.net/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html Also there is another effort made in a? B Sc's Thesis: http://www.emis.de/journals/NSJOM/Papers/30_3/NSJOM_30_3_123_130.pdf may be helpful, I haven't contact them, it should be helpful to ask. ? --- El dom, 28/12/08, Martin Bishop wrote: De: Martin Bishop Asunto: [M3devel] Arbitrary precision integers Para: m3devel at elegosoft.com Fecha: domingo, 28 diciembre, 2008 5:14 Does Modula-3 (or one of it's implementations) support arbitrary precision integers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 28 23:50:37 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:50:37 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081228100334.C41F3B04048@birch.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> Message-ID: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> This change makes me uneasy -- do you mean to say we have a uniform definition of sem_t when the different targets have different definitions? :-O On 28 Dec 2008, at 11:03, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/28 11:03:34 > > Modified files: > cm3/m3-libs/m3core/src/unix/Common/: m3makefile > cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile > cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile > Added files: > cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 > Removed files: > cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 > cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 > > Log message: > another common header file -- however this bought by defining sem_t > to have the maximum size and alignment of any system; > Solaris's 48 byte 64bit aligned version > The CM3 code base has one static sem_t in the pthread code. From hosking at cs.purdue.edu Sun Dec 28 23:46:44 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:46:44 +1100 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> The language itself does not support such things, though there may be a library implementing them somewhere. On 29 Dec 2008, at 09:14, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? From jay.krell at cornell.edu Mon Dec 29 00:49:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:49:25 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. There is only one sem_t in the system. It is global. This change just makes it usually larger and more aligned than necessary. The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. Usysdep.i3 seems best? - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jkrell at elego.de > Date: Mon, 29 Dec 2008 09:50:37 +1100 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > This change makes me uneasy -- do you mean to say we have a uniform > definition of sem_t when the different targets have different > definitions? :-O > > On 28 Dec 2008, at 11:03, Jay Krell wrote: > >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/28 11:03:34 >> >> Modified files: >> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >> Added files: >> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >> Removed files: >> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >> >> Log message: >> another common header file -- however this bought by defining sem_t >> to have the maximum size and alignment of any system; >> Solaris's 48 byte 64bit aligned version >> The CM3 code base has one static sem_t in the pthread code. > From jay.krell at cornell.edu Mon Dec 29 00:59:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:59:58 +0000 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> References: <4957FA36.8080308@bellsouth.net> <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> Message-ID: > though there may be > a library implementing them somewhere. The compiler itself has three such libraries embedded within it. :) Depending on how you define things. The front end cm3 does at least 64 bit integer math via 16 bit integers. cm3cg uses gmp and mpfr. The goals are probably the same for all three -- evaluate constant expressions correctly for the target. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: martinbishop at bellsouth.net > Date: Mon, 29 Dec 2008 09:46:44 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Arbitrary precision integers > > The language itself does not support such things, though there may be > a library implementing them somewhere. > > On 29 Dec 2008, at 09:14, Martin Bishop wrote: > >> Does Modula-3 (or one of it's implementations) support arbitrary >> precision integers? > From jay.krell at cornell.edu Mon Dec 29 00:55:16 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:55:16 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Note that I've been avoiding touching "mainstream" or already existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. Perhaps I should go ahead and cross that line, as my testability/testing allows? Slightly linked question is if user threads should remain. They don't work, I believe on any platform, but I understand could easily be made to. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; jkrell at elego.de > Date: Sun, 28 Dec 2008 23:49:25 +0000 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > > This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. > There is only one sem_t in the system. > It is global. This change just makes it usually larger and more aligned than necessary. > > > The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. > > > In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. > Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. > > > Usysdep.i3 seems best? > > > - Jay > > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jkrell at elego.de >> Date: Mon, 29 Dec 2008 09:50:37 +1100 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> This change makes me uneasy -- do you mean to say we have a uniform >> definition of sem_t when the different targets have different >> definitions? :-O >> >> On 28 Dec 2008, at 11:03, Jay Krell wrote: >> >>> CVSROOT: /usr/cvs >>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>> >>> Modified files: >>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>> Added files: >>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>> Removed files: >>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>> >>> Log message: >>> another common header file -- however this bought by defining sem_t >>> to have the maximum size and alignment of any system; >>> Solaris's 48 byte 64bit aligned version >>> The CM3 code base has one static sem_t in the pthread code. >> From hosking at cs.purdue.edu Mon Dec 29 01:14:02 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 11:14:02 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: User threads used to work on Solaris (since it uses getcontext/ setcontext to do thread switching instead of setjmp/longjmp) and these should work similarly on Linux/BSD/OSX. The only problem is that we need to use makecontext on some platforms to get a proper basis context for new threads (as opposed to using the hackier code that is there in ThreadPosix.m3. On 29 Dec 2008, at 10:55, Jay wrote: > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my testability/ > testing allows? > > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; jkrell at elego.de >> Date: Sun, 28 Dec 2008 23:49:25 +0000 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> >> This one I can undo and still feel ok -- just push sem_t into >> Utypes.i3 and expose it from a common Usem.i3. >> There is only one sem_t in the system. >> It is global. This change just makes it usually larger and more >> aligned than necessary. >> >> >> The danger would be if sem_t is ever embedded in something else OS- >> defined, or if the OS ever returned a pointer to one that Modula-3 >> code copied. >> >> >> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >> system-dependent stuff. >> Even existant Utypes.i3 have much redundancy -- all the ulong/ >> ushort/int8/int16/int32/int64 types are always the same, just stuff >> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >> should always be INT64 or UINT64. >> >> >> Usysdep.i3 seems best? >> >> >> - Jay >> >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jkrell at elego.de >>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> This change makes me uneasy -- do you mean to say we have a uniform >>> definition of sem_t when the different targets have different >>> definitions? :-O >>> >>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>> >>>> CVSROOT: /usr/cvs >>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>> >>>> Modified files: >>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>> Added files: >>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>> Removed files: >>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>> >>>> Log message: >>>> another common header file -- however this bought by defining sem_t >>>> to have the maximum size and alignment of any system; >>>> Solaris's 48 byte 64bit aligned version >>>> The CM3 code base has one static sem_t in the pthread code. >>> From jay.krell at cornell.edu Mon Dec 29 01:21:17 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 00:21:17 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Understood. You (someone) also has to fix the initialization. I think when you (you) moved the various user thread globals (?), such as the handler chain, into one nice thread object, "switchable" via just one pointer, it broke. I didn't look at the change in depth, I'm making a leap in my description of what you did. It is a /very/ reasonable design (or even likely The Correct Design), to collect up all the globals that represent a "thread" into one heap allocated struct...except it is an "object". I don't have SOLgnu/sun handy, this week, but can try it next week. I expect it fails as PPC_LINUX was. I THINK the "object" needs to be changed to a "traced ref" and then it can be initialized using, maybe, a preallocated global, for the first thread, or at least using regular "Malloc" and not going through the garbage collection...but I'm really just not sure. If I get X86_MSDOS/DJGPP up and going, I'll be a bit more motivated..but I think I'll so I386_SOLARIS first, since it is perhaps more modern/mainstream. You know, I tried having it NEW the object on-demand, but I think that ended up recursing to about the same place -- PushEFrame. It's "difficult" to repro now for me, this week, based on hardware/OS availability,because I ripped out PPC_LINUX user thread support (unless I install NetBSD/FreeBSD in a VM or use birch, all of which are, honestly, easy). - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 11:14:02 +1100 > CC: jkrell at elego.de; m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > User threads used to work on Solaris (since it uses getcontext/ > setcontext to do thread switching instead of setjmp/longjmp) and these > should work similarly on Linux/BSD/OSX. The only problem is that we > need to use makecontext on some platforms to get a proper basis > context for new threads (as opposed to using the hackier code that is > there in ThreadPosix.m3. > > On 29 Dec 2008, at 10:55, Jay wrote: > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? >> >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. >> >> - Jay >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: hosking at cs.purdue.edu; jkrell at elego.de >>> Date: Sun, 28 Dec 2008 23:49:25 +0000 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> >>> This one I can undo and still feel ok -- just push sem_t into >>> Utypes.i3 and expose it from a common Usem.i3. >>> There is only one sem_t in the system. >>> It is global. This change just makes it usually larger and more >>> aligned than necessary. >>> >>> >>> The danger would be if sem_t is ever embedded in something else OS- >>> defined, or if the OS ever returned a pointer to one that Modula-3 >>> code copied. >>> >>> >>> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >>> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >>> system-dependent stuff. >>> Even existant Utypes.i3 have much redundancy -- all the ulong/ >>> ushort/int8/int16/int32/int64 types are always the same, just stuff >>> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >>> should always be INT64 or UINT64. >>> >>> >>> Usysdep.i3 seems best? >>> >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> From: hosking at cs.purdue.edu >>>> To: jkrell at elego.de >>>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>>> Subject: Re: [M3commit] CVS Update: cm3 >>>> >>>> This change makes me uneasy -- do you mean to say we have a uniform >>>> definition of sem_t when the different targets have different >>>> definitions? :-O >>>> >>>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>>> >>>>> CVSROOT: /usr/cvs >>>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>>> >>>>> Modified files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>>> Added files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>>> Removed files: >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>>> >>>>> Log message: >>>>> another common header file -- however this bought by defining sem_t >>>>> to have the maximum size and alignment of any system; >>>>> Solaris's 48 byte 64bit aligned version >>>>> The CM3 code base has one static sem_t in the pthread code. >>>> > From lemming at henning-thielemann.de Mon Dec 29 01:42:24 2008 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 29 Dec 2008 01:42:24 +0100 (CET) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: On Sun, 28 Dec 2008, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? http://modula3.elegosoft.com/cm3/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html It's not well optimized and maybe not enough tested. If you find it too slow, you might write wrapper to the GNU multiprecision library GMP. Maybe SWIG can help writing the wrapper. (or you try Haskell, which also uses GMP :-) From wagner at elegosoft.com Mon Dec 29 08:46:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 29 Dec 2008 08:46:55 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Quoting Jay : > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my > testability/testing allows? I wanted to comment on this before. I've no objection at trying to remove duplicated code as far as possible, but when changing system interfaces (Uxxx.is), please make sure that they really work on the target platform in question; if possible, add some tests. > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. I was not aware that user threads are broken; why? Anyway, this code should not be removed; I was rather hoping for a compile-time option to switch between user and system threads in cm3. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Mon Dec 29 09:16:13 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 19:16:13 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Message-ID: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > Quoting Jay : > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? > > I wanted to comment on this before. I've no objection at trying to > remove duplicated code as far as possible, but when changing system > interfaces (Uxxx.is), please make sure that they really work on the > target platform in question; if possible, add some tests. > >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. > > I was not aware that user threads are broken; why? > Anyway, this code should not be removed; I was rather hoping for a > compile-time option to switch between user and system threads in cm3. Definitely doable! I've been meaning to do this for a long time now. > > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From jay.krell at cornell.edu Mon Dec 29 09:47:02 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 08:47:02 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> Message-ID: cm3 -DNOPTHREAD compiles m3core for user threads. I /sort of/ (almost) haven't removed anything. I did remove PPC_LINUX stuff. In new platforms (openbsd, {amd64,sparc}_linux, cygwin), I might have initially added partial support for user threads, never complete, tested, working support, and then removed it. User threads are broken because PushEFrame uses uninitialized NULL self. PushEFrame is always called before ThreadF__Init (which initializes self), but the other two threading libraries (which don't have "self" at all) initialize on demand what they need in PushEFrame (the thread local handler stack). I think self/Thread.T needs to be a traced ref, instead of an object, so the initial value can be to a preallocated static/global. I'm not sure of the language here. "object" seems to deprive you of the "value type". Whereas refs do not. I tried allocating self on demand in PushEFrame but that didn't work. It ended up back in PushEFrame. I tried more tactics after that, but then gave up. Or move the handle stack out of Thread.T and into a separate global like it was. Either change would not affect pthreads or NT threads. I would like to see how much I can whittle down m3core of platform-specific stuff, and then maybe see about adding user thread support for the platforms that never had it. I'll add I386_SOLARIS soon, and don't want to duplicate a "large" directory of files as part of that. I really dislike the multiplicity of such a system. I have another chunk of m3core about to go though, for systems that don't have user threads. Maybe not one that is worthwhile. I'm still not much of a believer in user threads. Essentially I have always had kernel threads, with a processor-portable interface, and one machine-wide scheduler..seems good.. Every thread mechanism needs its own locking mechanism, which won't interace with any other thread or locking mechanism. Kind of like picking a C compiler or library -- you want there to be just one obvious choice per-platform and have everyone use it, so that there is interop and better overall scheduling. You know..Modula-3 I think has problems when the executable doesn't use the Modula-3 runtime and/or when threads are created outside the runtime, at then run Modula-3 code. That seems unfortunate. I /should/ be able to write a "plugin .dll" in Modula-3 and have it loaded by a C or C++ application, and have garbage collection work. Win32 is also kind of wonky here, in that it has "fibers". Fibers are like swapcontext, plus the stack allocation is handled for you. Anyone using them has to implement their own scheduler and their own locking mechanisms. Win32 critical sections, mutexes, etc. don't interact "correctly" with them. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: wagner at elegosoft.com > Date: Mon, 29 Dec 2008 19:16:13 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > >> Quoting Jay : >> >>> >>> Note that I've been avoiding touching "mainstream" or already >>> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >>> >>> Perhaps I should go ahead and cross that line, as my testability/ >>> testing allows? >> >> I wanted to comment on this before. I've no objection at trying to >> remove duplicated code as far as possible, but when changing system >> interfaces (Uxxx.is), please make sure that they really work on the >> target platform in question; if possible, add some tests. >> >>> Slightly linked question is if user threads should remain. >>> They don't work, I believe on any platform, but I understand could >>> easily be made to. >> >> I was not aware that user threads are broken; why? >> Anyway, this code should not be removed; I was rather hoping for a >> compile-time option to switch between user and system threads in cm3. > > Definitely doable! I've been meaning to do this for a long time now. > >> >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, >> Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 >> 45 86 95 >> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: >> Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: >> DE163214194 >> > From mika at async.caltech.edu Mon Dec 29 10:23:17 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 29 Dec 2008 01:23:17 -0800 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." Message-ID: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Jay writes: > ... > >I'm still not much of a believer in user threads. >Essentially I have always had kernel threads, with a processor-portable >interface, and one machine-wide scheduler..seems good.. > ... Except that on some machines, some operating systems, Modula-3's user threads are much more efficient (and if not, far more easily modifiable) than the provided kernel threads when you perform certain types of operations in certain situations. I often write programs with many (hundreds, thousands) of active threads and run into these sorts of problems frequently. You could argue that it should not be so... that every OS should provide kernel threads that are better than Modula-3's own user threads under all conceivable workloads, but what good does that do? Seems better to have a way of avoiding that can of worms by being able to control one's own implementation. Another reason that occurred to me when reading some very old M3 documentation is that there do occasionally exist systems that don't have a real operating system. There may also be other deficient systems that don't have pthreads. With an existing user-level threading library you can easily support such systems. (M3 has supported such systems in the past...) Mika From jay.krell at cornell.edu Mon Dec 29 10:47:26 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 09:47:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> References: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Message-ID: I'm aware of systems without threads, e.g. MS-DOS or apparently many of the systems Modula-3 first ran on. (Really? pthreads wasn't very available back then?) The case of many threads, um.. You don't run out of address space on the stacks? I guess the Modula-3 stuff honors requests for very small stacks? (need to fix that it leaks a pthread_attr here..) Not a problem on 64bit systems. The large number of threads doesn't bog down scheduling? Is it hundreds/thousands of threads alive at the same time, or serially? Serially, no big deal. Modifiable, sure. The more you implement "yourself", the more you can change. You can hope for folks to build in some "configurability" and "tunability", but nothing beats the ability to change or replace the thing arbitrarily. Maybe having hundreds/thousands is "wrong"? I have repeatedly seen guidance against creating "many" threads. Better to have a "thread pool", and/or use "completion ports" (aka select?). Maybe you should have just a few threads, and then decide for yourself what each should run? You know, isn't the guidance to have roughly one thread per processor? You aren't going to scale beyond that..unless you dedicate a thread per slow i/o, which is wasteful. You know there is this whole thing where..too many cooks spoil the broth. Having multiple schedulers on the system means they won't do an ideal job. It is best to have one scheduler, that "knows everything" -- knows about all the threads and what they are waiting for. The cost of threads..is both low and high. - Jay ---------------------------------------- > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 01:23:17 -0800 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > Jay writes: >> > ... >> >>I'm still not much of a believer in user threads. >>Essentially I have always had kernel threads, with a processor-portable >>interface, and one machine-wide scheduler..seems good.. >> > ... > > Except that on some machines, some operating systems, Modula-3's > user threads are much more efficient (and if not, far more easily > modifiable) than the provided kernel threads when you perform certain > types of operations in certain situations. I often write programs > with many (hundreds, thousands) of active threads and run into these > sorts of problems frequently. > > You could argue that it should not be so... that every OS should > provide kernel threads that are better than Modula-3's own user > threads under all conceivable workloads, but what good does that > do? Seems better to have a way of avoiding that can of worms by > being able to control one's own implementation. > > Another reason that occurred to me when reading some very old > M3 documentation is that there do occasionally exist systems > that don't have a real operating system. There may also be > other deficient systems that don't have pthreads. With an existing > user-level threading library you can easily support such systems. > (M3 has supported such systems in the past...) > > Mika From jay.krell at cornell.edu Tue Dec 30 03:07:24 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:07:24 +0000 Subject: [M3devel] current tinderbox status, broken by me? Message-ID: NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho. - Jay From stsp at elego.de Tue Dec 30 03:16:46 2008 From: stsp at elego.de (Stefan Sperling) Date: Tue, 30 Dec 2008 03:16:46 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229203344.692E1B04044@birch.elegosoft.com> References: <20081229203344.692E1B04044@birch.elegosoft.com> Message-ID: <20081230021646.GA8558@jack.stsp.name> On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/29 21:33:44 Hey Jay, > Log message: > enough I386_OPENBSD support to build a bootstrap package, > that builds on the target, and results in a cm3 that > runs and correct errors out for lack of cm3.cfg, Cool, I'd like to play with this, once you have a bootstrap binary. Stefan From jay.krell at cornell.edu Tue Dec 30 03:28:09 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:28:09 +0000 Subject: [M3devel] regular cross builds, package format? Message-ID: So..it is reasonably easy to do a cross build, that outputs a bunch of assembly files, some C files, a "script" or makefile to build them. What you (I) do with such output is copy it to the target machine and run the script, which requires a C compiler, linker, and whatever libraries, and whatever "script runner" (sh, make, etc.). I've been using this stuff to bring up new platforms. It works well enough. There are a number of obvious questions. Should the tinderbox go ahead and do that, for "every" target platform? ("every" defined as known working ones) What should the form of the "cross build archive" be? Or, for that matter, what should be form of "native distributions"? Should it be possible to build one "intermediate" distribution that can be "finished" on either any autoconf-able Posix platform, or any supported platform? That is, should the platform-specificity be factored out of the intermediate representation, and the cross or universal or even native (unlikely) format be that of portable intermediate files, and the "finish" step includes building the backend (not fast). I gather this last thing was done by PM3, though I also gather it isn't a small task. (I need to look at their code.) Platform-specifity leaks into the intermediate representation presumably a lot. You know, the size of integers, alignment affects on layout, etc. As well, if some systems use RTOSmmap.c and others RTOSbr.c -- or even .m3 files, the "universal" distribution would have to carry both files and delay the choice. If you reject the "universal" notion, there remain some questions, though less complicated ones. Simpple ones like what the "finish script" is. I generate a makefile, but it seems to be buggy. Sometimes there is no incrementality, it builds everything every time. Sometimes it generates a warning for each file. I don't know why, but since these are so far only for my use, ok. Of course there remains the question of cminstall or no cminstall. I've been getting by without cminstall for quite some time across a few platforms. A "universal" distribution would probably chose neither of the current options, but something that works better like autoconf. I'm not a huge fan of autoconf, but it has some very tempting characteristics, where it can sniff out system behavior as long as it is either in a known set or not too far off. - Jay From jay.krell at cornell.edu Tue Dec 30 03:57:12 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:57:12 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081230021646.GA8558@jack.stsp.name> References: <20081229203344.692E1B04044@birch.elegosoft.com> <20081230021646.GA8558@jack.stsp.name> Message-ID: Cool. You can try http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-I386_OPENBSD-1.tar.gz, but hopefully soon I'll have "std" and "min". The boot archive builds cm3 only, no libs, no cm3cg -- though it should be enough to build them natively (plus a current source tree). If you use this, be aware that something is a little off in my bootstrapping process in that none of do-cm3-std do-cm3-min upgrade etc., seem quite right. upgrade in particular requires pkg/m3core and pkg/libm3. None of the sets imply TO ME building both m3cc and cm3, though "front" might actually (it ought not build m3cc). Also, the sorting to build in dependency order in pylib.py seems to not be working, strange. Anyway, this is fairly minor. There is also a definite possiblity of cross-building a complete distribution, you just have to build GNU cc (similar to building m3cg), GNU as, GNU ld, and either copy or build the C runtime and startup files...at least for *_*BSD, *_LINUX, SOLgnu, I386_CYGWIN, I don't know about the others (Darwin does not use GNU ld nor I think GNU as, whatever it does use, I don't know if it is easily re-hosted, similar SOLsun.) (Tony, please speak up if you really meant X86_* and not I386_*.) - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 03:16:46 +0100 > From: stsp at elego.de > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/29 21:33:44 > > Hey Jay, > >> Log message: >> enough I386_OPENBSD support to build a bootstrap package, >> that builds on the target, and results in a cm3 that >> runs and correct errors out for lack of cm3.cfg, > > Cool, I'd like to play with this, once you have a bootstrap binary. > > Stefan From dabenavidesd at yahoo.es Tue Dec 30 07:01:45 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 30 Dec 2008 06:01:45 +0000 (GMT) Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: Message-ID: <908914.75062.qm@web28601.mail.ukl.yahoo.com> In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? I would like to hear your comments on that, let me know what else is missing to know the FilePosix is safe. Thanks in advance, --- El lun, 29/12/08, Jay wrote: De: Jay Asunto: [M3devel] current tinderbox status, broken by me? Para: "m3devel" Fecha: lunes, 29 diciembre, 2008 9:07 NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 30 08:48:17 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:48:17 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <908914.75062.qm@web28601.mail.ukl.yahoo.com> References: <908914.75062.qm@web28601.mail.ukl.yahoo.com> Message-ID: The C code I introduced should behave identically to or better than the Modula-3 code it replaced. The Modula-3 code dependend on tedious error-prone rewriting of C headers in Modula-3. Mistakes there easily lead to the overall safety of the system being violated. I believe the C code adheres to the safety guarantee below. Can you check? - Jay ________________________________ > Date: Tue, 30 Dec 2008 06:01:45 +0000 > From: dabenavidesd at yahoo.es > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] current tinderbox status, broken by me? > > In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." > Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? > For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? > I would like to > hear your comments on that, let me know what else is missing to know the FilePosix is safe. > Thanks in advance, > > --- El lun, 29/12/08, Jay wrote: > De: Jay > Asunto: [M3devel] current tinderbox status, broken by me? > Para: "m3devel" > Fecha: lunes, 29 diciembre, 2008 9:07 > > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix..m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 > -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to clashing > definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I > had another new platform defined in one or another file and then when I > backed down and just did one reasonly finished platform, I forgot at first to > cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to run > into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, additions > are safe" annotation? > Probably not. It's probably too gray and more complicated than that, since > additions in general are not as safe as people thing. Adding to a record -- > folks might be copying based on the size and might get back an older smaller > version. Adding to an enum, folks might be switching on it and expect to have > the range covered. Adding a function to an interface though, probably ok. Not > the case here, and perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an > enum, and such annotated > types, switches on them must have an else clause, and they must not be used for > array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf is > recorded in both Csetjmp.i3 and Target..i3. Duplication is generally bad imho > > From jay.krell at cornell.edu Tue Dec 30 08:51:56 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:51:56 +0000 Subject: [M3devel] darwin-generic waitpid? Message-ID: It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and having a C wrapper do the translation. As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. - Jay From jay.krell at cornell.edu Tue Dec 30 09:34:03 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 08:34:03 +0000 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the waitpid(nohang)/sleep() loop that I had removed in other places, alas -- replaced in pthreads by waitpid(hang), whereas userthreads still has waitpid(nohang)/sleep (another flaw in user threads?) and..isn't this a little odd: RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); If there are no users of this, I'll just comment it out. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; m3devel at elegosoft.com > Date: Tue, 30 Dec 2008 07:51:56 +0000 > Subject: [M3devel] darwin-generic waitpid? > > > It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and > having a C wrapper do the translation. > > > As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. > > > - Jay From wagner at elegosoft.com Tue Dec 30 11:01:24 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 30 Dec 2008 11:01:24 +0100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 for details. It occurs whenever you add or remove target platforms. It is exactly what the last-ok build was made for :-) As for your other comments, I would appreciate if the compiler would output the exact conflicting file paths, but I believe that the information is not easily available at that point. Olaf Quoting Jay : > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 30 11:04:23 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:04:23 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> References: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> Message-ID: Ah, I thought the Tinderbox always ran upgrade.sh first. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 11:01:24 +0100 > From: wagner at elegosoft.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] current tinderbox status, broken by me? > > This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 > for details. It occurs whenever you add or remove target platforms. > > It is exactly what the last-ok build was made for :-) > > As for your other comments, I would appreciate if the compiler would > output the exact conflicting file paths, but I believe that the > information is not easily available at that point. > > Olaf > > Quoting Jay : > >> >> NEXT Fatal Error: bad version stamps: SocketPosix.m3 >> 4678 >> 4679 version stamp mismatch: Compiler.Platform >> 4680 => SocketPosix.m3 >> 4681 => Compiler.i3 >> 4682 version stamp mismatch: Compiler.ThisPlatform >> 4683 => SocketPosix.m3 >> 4684 => Compiler.i3 >> 4685 NEXT *** execution of cm3 -build >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? failed *** >> 4686 compile return value: 0 >> 4687 [end compile 2008.12.30 02:25:17] >> 4688 *** COMPILE FAILED >> No More Errors >> >> >> 1) >> Is it reasonable to suggest that the compiler output the source to >> clashing definitions? >> Or is that a bogus naive suggestion? >> It is probably legitimate, at least to identify the files in question. >> Could be that not enough information is around to identify "what >> changed", just some "summary hash". >> >> >> 2) Is it really broken? >> I did go slightly round and round on this. >> I had another new platform defined in one or another file and then >> when I backed down and just did one reasonly finished platform, I >> forgot at first to cleanup all the debris. >> But it certainly worked when I commited. >> I'm progressing on the native build, so if it is broken, I expect to >> run into it myself. >> >> >> 3) Is it reasonable to annotate a type with some sort of "relax, >> additions are safe" annotation? >> Probably not. It's probably too gray and more complicated than that, >> since additions in general are not as safe as people thing. Adding >> to a record -- folks might be copying based on the size and might >> get back an older smaller version. Adding to an enum, folks might be >> switching on it and expect to have the range covered. Adding a >> function to an interface though, probably ok. Not the case here, and >> perhaps not something the compiler even complains about. >> One that that might be eek-out-able is adding to an enum, and such >> annotated types, switches on them must have an else clause, and they >> must not be used for array indexing? >> Eh, maybe just leave it alone. >> >> >> Anything else? Is this just basically how it has to be? >> OR, how about, can the duplicity be eliminated? >> Like, can Compiler.i3 be fully generated by the compiler? >> Similarly, Csetjmp.i3 should be generated by the compiler, eh? >> Or at least partly? You know, the size and alignment of the jmp_buf >> is recorded in both Csetjmp.i3 and Target.i3. Duplication is >> generally bad imho > > > > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > From jay.krell at cornell.edu Tue Dec 30 11:08:53 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:08:53 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: I wish: 1) you could put declarations in .m3 files 2) could be entirely optional, as long as the C functions are named Module__Function. There are three implementations of RTOS.GetMemory. In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 08:00:29 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/30 08:00:29 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > FilePosixC.c m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 > > Log message: > put the C functions in their own interface, and other small change > (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) > From jay.krell at cornell.edu Tue Dec 30 18:38:33 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 17:38:33 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: [was truncated, and now edited slightly] > I wish: > > > 1) you could put declarations in .m3 files > > > 2) could be entirely optional, as long as the C functions are named Module__Function. > > > There are three implementations of RTOS.GetMemory. > In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. > > > - Jay > > ---------------------------------------- >> Date: Tue, 30 Dec 2008 08:00:29 +0000 >> To: m3commit at elegosoft.com >> From: jkrell at elego.de >> Subject: [M3commit] CVS Update: cm3 >> >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/30 08:00:29 >> >> Modified files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 >> FilePosixC.c m3makefile >> Added files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 >> >> Log message: >> put the C functions in their own interface, and other small change >> (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) >> From hosking at cs.purdue.edu Wed Dec 31 00:47:46 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:47:46 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: Message-ID: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay From hosking at cs.purdue.edu Wed Dec 31 00:48:43 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:48:43 +1100 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: Yes, this definitely needs cleanup. On 30 Dec 2008, at 19:34, Jay wrote: > > On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the > waitpid(nohang)/sleep() loop that I had removed in other places, > alas -- replaced in pthreads by waitpid(hang), whereas userthreads > still has waitpid(nohang)/sleep (another flaw in user threads?) > > and..isn't this a little odd: > > RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); > > If there are no users of this, I'll just comment it out. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; m3devel at elegosoft.com >> Date: Tue, 30 Dec 2008 07:51:56 +0000 >> Subject: [M3devel] darwin-generic waitpid? >> >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. >> >> >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay From hosking at cs.purdue.edu Wed Dec 31 00:50:35 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:50:35 +1100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: These result from inconsistency between the platforms defined statically in cm3 and the platforms declared in m3core. You need to use the version in m3core that matches what you had when building cm3. Then, extend cm3 before compiling the new m3core version. On 30 Dec 2008, at 13:07, Jay wrote: > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3- > ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/ > birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho From jay.krell at cornell.edu Wed Dec 31 01:09:58 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 00:09:58 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: I'd like to avoid cloning any headers, and use clear portable C wrappers to bridge between Modula-3 and stuff written/declared in C. e.g. no bit fields. These are contrary goals. Or, at least, only clone headers that are essentially identical across all Posix platforms, or which the only variables are endianness and word size. There are a few possible compromises. Currently there is linux-common, openbsd-common. I would NOT like to add netbsd-common, freebsd-common, solaris-common. As such, I'm pushing stuff from "*-common" to plain "Common". As well, by not cloning header content that is unused in the cm3 tree, much is saved. Compare the number of lines in LINUXLIBC6 vs. linux-common. But I'd like to get further than that. Similarly, not cloning header content that doesn't present a portable interface, which tends to lead to it not being used, since most of the cm3 code is factored only on Posix vs. Win32, and not one Posix superset or another. I also believe that C is "everywhere" and will remain "everywhere". Except very constrained "safe" environments like a web browser or Java-only Google phone, where Modula-3 is about as far as being able to run as C is. I think C wrappers are very preferable to cloning headers. The endian-ness is actually abstracted out in normal C code. There's the various macros for cracking the 32 bit status. Ok, maybe that is apples and oranges. It is abstracted out in the client C code and the client Modula-3 code, except that Modula-3 client code uses LOOPHOLE. Both the C headers and Modula-3 headers are "messy", the C headers sometimes more so, such as to cater to other (BSD) interfaces. I realize there are the Win32 headers to be concerned with as well. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 10:47:46 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > On 30 Dec 2008, at 18:51, Jay wrote: > >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. > > I am concerned that so much C wrapper stuff is starting to creep in. > I want to be able to see the endianness in the Modula-3 code, not have > it squirrelled away in C code. What if I don't have a C compiler? > Jay, please be careful with your assumptions about platforms always > having C. > >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay > From hosking at cs.purdue.edu Wed Dec 31 01:47:48 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 11:47:48 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Sure, I understand. I think what you are doing is great for improving the reliability of the C-interface code. Are you able to test all the platforms? I am worried we are going to have bugs introduced that someone (you!) will need to chase down. On 31 Dec 2008, at 11:09, Jay wrote: > > I'd like to avoid cloning any headers, and use clear portable C > wrappers to bridge between Modula-3 and stuff written/declared in C. > e.g. no bit fields. > > These are contrary goals. > > Or, at least, only clone headers that are essentially identical > across all Posix platforms, or which the only variables are > endianness and word size. > > There are a few possible compromises. > Currently there is linux-common, openbsd-common. > I would NOT like to add netbsd-common, freebsd-common, solaris-common. > As such, I'm pushing stuff from "*-common" to plain "Common". > > As well, by not cloning header content that is unused in the cm3 > tree, much is saved. > Compare the number of lines in LINUXLIBC6 vs. linux-common. > But I'd like to get further than that. > > Similarly, not cloning header content that doesn't present a > portable interface, which tends to lead to it not being used, since > most of the cm3 code is factored only on Posix vs. Win32, and not > one Posix superset or another. > > I also believe that C is "everywhere" and will remain "everywhere". > Except very constrained "safe" environments like a web browser or > Java-only Google phone, where Modula-3 is about as far as being able > to run as C is. > > I think C wrappers are very preferable to cloning headers. > > The endian-ness is actually abstracted out in normal C code. > There's the various macros for cracking the 32 bit status. > Ok, maybe that is apples and oranges. It is abstracted out in the > client C code and the client Modula-3 code, except that Modula-3 > client code uses LOOPHOLE. Both the C headers and Modula-3 headers > are "messy", the C headers sometimes more so, such as to cater to > other (BSD) interfaces. > > I realize there are the Win32 headers to be concerned with as well. > > - Jay > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jay.krell at cornell.edu >> Date: Wed, 31 Dec 2008 10:47:46 +1100 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] darwin-generic waitpid? >> >> On 30 Dec 2008, at 18:51, Jay wrote: >> >>> >>> It looks like waitpid's output is endian-dependent, and that >>> therefore darwin-generic is wrong on some architectures, asis linux- >>> common and openbsd-common. >>> >>> >>> I expect to fix this shortly, by replacing all the Modula-3 result >>> bitfields with uint8_t and >>> having a C wrapper do the translation. >> >> I am concerned that so much C wrapper stuff is starting to creep in. >> I want to be able to see the endianness in the Modula-3 code, not >> have >> it squirrelled away in C code. What if I don't have a C compiler? >> Jay, please be careful with your assumptions about platforms always >> having C. >> >>> As well, for increased portability, the flags will be input uint8_t >>> fields in the record, rather than platform-dependent flags. >>> >>> >>> - Jay >> From jay.krell at cornell.edu Wed Dec 31 02:25:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 01:25:31 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: I am not able to test everything this week, on vacation, just have two laptops with me, and not quite /everything/ at home yet, but I have available a lot now and am still working on getting and setting up more hardware and operating systems, and maybe will use virtualization more (I386_OPENBSD is in VirtualPC, Solaris/x86 in VirtualBox bugchecks the host too frequently(. I'm not sure the inventory is important or impressive or embarassing. :) I've got a variety of MIPS64 (SGI), PPC64 (RS/6000), PPC32 (Apple), ARM (Western Digital MyBook "World Edition), IA64 (HP Integrity), SPARC64 (Sun), AMD64 (Sun and PCs), running variety of Linux, OpenBSD, Darwin, AIX, not currently NetBSD but will, NT, Solaris.. Alphas are on the way, HP-PA might be on the way, the main missing piece is PPC64_DARWIN hardware. :) - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 05:32:25 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 04:32:25 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: Tony, Hi, sorry, I'm a little confused. Beware C wrappers -- don't write them. That's what it sounded like at first. Or: Beware C wrappers -- be careful writing them. ? On the second, well, yeah, ok. I haven't actually yet improved much -- since I stick to "new" ports. "Soon" I might take larger steps. I'll probably first cave and add user thread support to {PPC,AMD64,SPARC{32,64}}_LINUX, I386_SOLARIS, *_OPENBSD, etc., while continuing to minimize or eliminate header clones. And then maybe move LINUXLIBC6 to linux-common/common. I should say..um..IF your concern is just "be careful writing C code"... I wonder how far I should go. I'm obessive. I keep looking at linux-common/openbsd-common/cygwin and I wonder what I can remove. And how. Removing unused stuff, easy. Wrapping functions that actually are functions, not too bad. But types and constants..you can try to write functions to take over all their uses, but I don't think that will do for all of them. For example, struct flock I removed. The "worst" cases are probably: Uerrno Upthread In particular, these are the files for which C wrapping is likely to do the "most damage" with respect to efficiency and code structure. Uerrno..the way to wrap it in C is, well..one thing I do NOT want to do is go through and make up our own errno values and map back and forth everywhere. Yuck. However, one thing that seems half reasonable is to have GetEAGAIN(), GetEINVALID(), GetENOMEM(), etc. or a bulk struct { int EINVAL, ENOMEM, etc. } * GetErrnoValues(). The second version is "less chatty" but risky, like if the struct gets copied and changes size through time (versioning). The first is much safer. In both cases, the errors loses their const-ness and you can, I assume, no longer switch/case on them. Now, error checking..efficiency..probably ok -- code that is dealing with errors doesn't have to be fast. Change the switches/cases to if/elseif/elseif ladders. I think I will do this. Upthread, here I think part of the answer is that all of the functions have identical prototypes. So "half" the file just goes to Common and done, no C wrapping. That leaves the types. sem_t, which is vaguely related, there is just one of. I think we can just declare that variable in C, since you don't like the over allocation/alignment. But that's an unusual benefit -- something there is only one of. For types that are created repeatedly..well, it depends. I have to look into the allocator interface.. if the objects are heap allocated, one thing you can maybe do is like: RTAllocator(Upthread.GetSize_rwlock(), Upthread.GetAlign_rwlock()). Assuming size and alignment are parameters, or assuming that alignment is always maximal. If the objects are stack allocated, well, that breaks down. You /could/ heap allocate them always, but that's a sort of inefficiency I'm reluctant to introduce. I don't mind turning constants, esp. seldom used ones, into function calls, or copying small structs (stat), but turning stack allocation into heap allocation, yuck. Another answer is to leave these alone. Generate Uerrno.i3 from Uerrno.i3.cpp and leave it that. Maybe generate them as a matter of course in any /native/ build and compare to the "checked in" copy. Well, um, native builds also benefit from being able to run tests, so you can accomplish the same thing there -- pass the constants, type sizes, etc. to C code that asserts equivalence. Upthread.i3 could be similar. To address the types vs. prototypes, I'd say move the types, and probably all the initializers, into a per-system Upthreadtypes.i3, or Usysdep.i3. Usocket.i3 is similar -- identical prototypes, but constants vary. Move the constants into "Usysdep.i3"? Or use accessor functions? I have to look at the code more besides. I don't have a plan. Anyway, at the very least, we have: jay at ppca:/dev2/cm3/m3-libs/m3core/src/unix$ wc -l linux-common/*.i3 415 total openbsd-common => 516 wc -l linux-libc6/*.i3 4011 Not bad to be at 10%, and user threads won't regress all that much. Maybe stop there? Well, Uutsname.i3 is probably going away. It has just one user. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 06:33:22 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 05:33:22 +0000 Subject: [M3devel] variations of waitpid..? Message-ID: variations of waitpid.. Hey, we already have, that I put in on my initiative, months ago: C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) VAR result: int; statusM3: Uwaitpid.waitpid_status_t; CONST Delay = 0.1D0; BEGIN LOOP result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Pause(Delay); END; 0 *> RETURN statusM3.w_Loophole; END WaitProcess; pthreads and cygwin: C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 and C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) VAR statusM3: Uwaitpid.waitpid_status_t; BEGIN LOOP WITH r = Uwaitpid.waitpid(pid, statusM3) DO IF r> 0 THEN RETURN statusM3.w_Loophole; END; < 0*> END; END; END WaitProcess; win32: PROCEDURE WaitProcess (pid: int): int = BEGIN RETURN 0; END WaitProcess; ThreadPThread and WaitProcessCygwin should share code, but that's not the point. However, now, to fix sysutils, we need something LIKE: UNSAFE INTERFACE SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; BEGIN END SystemPosixWaitPid. pthreads: UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = BEGIN RETURN waitpid(pid, status, 0); END WaitPid; BEGIN END SystemPosixWaitPidEfficient. user threads: UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = VAR result: int; CONST Delay = 0.1D0; BEGIN LOOP result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Thread.Pause(Delay) END; END WaitPid; BEGIN END SystemPosixWaitPidPause. questions: Aside: Where do I put "unsafe"? Relevant: Sysutils has to implement this itself, right? Due to bootstrapping concerns. But we should still put /something/ in m3core, right? So then the big question, what to call it? Probably, the existing WaitProcess that does more than this WaitPid, should be scaled back and only do what WaitPid does? I will likely commit exactly what is shown above. The new interface and modules are in sysutils. The interface is private. The unfinished part is putting something in m3core, that sysutils could use, if not for boostrapping issuers, that it could use in the future. I kind of thing something a bit more is needed. You know, the Win32 implementation that asserts(false) is suspicious. At the very least, it should be removed. I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 interfaces? I also suspect part of the problem is that sysutils implements to a large extent the same abstractions as m3core and/or libm3, but with variations, causing it to be unable to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions maybe are not general/featureful enough. (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) ?? - Jay From jay.krell at cornell.edu Wed Dec 31 08:13:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 07:13:02 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: References: Message-ID: Hm. How about: PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? ? true for pthreads/win32/cygwin false for userthreads duplicated privately in sysutils to avoid depending on newer m3core like I did for the portable C-wrapped waitpid ? That way..the other code can do whatever it already is doing, no need to come up with abstractions for looping with nohang, vs. that plus the status loophole. The older fix I put it can be shrunk. The name of this is debatable, but I think the minimal level of abstraction is goodness. The name is actually perhaps too..complicated..like, what does it mean? Who uses it? What do they do with it? It means that waiting on a process to end allows threads in the waiting process to run. Perhaps it is too low level. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Wed, 31 Dec 2008 05:33:22 +0000 > Subject: [M3devel] variations of waitpid..? > > > variations of waitpid.. > > Hey, we already have, that I put in on my initiative, months ago: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) > VAR > result: int; > statusM3: Uwaitpid.waitpid_status_t; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); > IF result # 0 THEN EXIT END; > Pause(Delay); > END; > 0 *> > RETURN statusM3.w_Loophole; > END WaitProcess; > > pthreads and cygwin: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 > and > C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) > VAR > statusM3: Uwaitpid.waitpid_status_t; > BEGIN > LOOP > WITH r = Uwaitpid.waitpid(pid, statusM3) DO > IF r> 0 THEN > RETURN statusM3.w_Loophole; > END; > < 0*> > END; > > END; > END WaitProcess; > > win32: > PROCEDURE WaitProcess (pid: int): int = > BEGIN > > RETURN 0; > END WaitProcess; > > ThreadPThread and WaitProcessCygwin should share code, but that's not the point. > > However, now, to fix sysutils, we need something LIKE: > > UNSAFE INTERFACE SystemPosixWaitPid; > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; > > BEGIN > END SystemPosixWaitPid. > > pthreads: > > UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > BEGIN > RETURN waitpid(pid, status, 0); > END WaitPid; > > BEGIN > END SystemPosixWaitPidEfficient. > > user threads: > > UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > VAR result: int; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); > IF result # 0 THEN > EXIT > END; > Thread.Pause(Delay) > END; > END WaitPid; > > BEGIN > END SystemPosixWaitPidPause. > > questions: > > Aside: Where do I put "unsafe"? > > Relevant: Sysutils has to implement this itself, right? > Due to bootstrapping concerns. > > But we should still put /something/ in m3core, right? > > So then the big question, what to call it? > > Probably, the existing WaitProcess that does more than this WaitPid, should be scaled > back and only do what WaitPid does? > > I will likely commit exactly what is shown above. > The new interface and modules are in sysutils. The interface is private. > > The unfinished part is putting something in m3core, that sysutils could use, if > not for boostrapping issuers, that it could use in the future. > > I kind of thing something a bit more is needed. > You know, the Win32 implementation that asserts(false) is suspicious. > At the very least, it should be removed. > > I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. > Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 > interfaces? > > I also suspect part of the problem is that sysutils implements to a large extent > the same abstractions as m3core and/or libm3, but with variations, causing it to be unable > to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions > maybe are not general/featureful enough. > > (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) > > > ?? > > - Jay From mika at async.caltech.edu Wed Dec 31 08:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 30 Dec 2008 23:22:41 -0800 Subject: [M3devel] variations of waitpid..? In-Reply-To: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." Message-ID: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> I don't think it should be possible to stop other threads if the runtime could even conceivably support an environment where that's not necessary. You're introducing dependencies that will lead to (undetectable) deadlocks. People who declare their own <*EXTERNAL*> things are, as usual, on their own. Mika Jay writes: > >Hm. How about: > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? > >? >true for pthreads/win32/cygwin >false for userthreads > >duplicated privately in sysutils to avoid depending on newer m3core > like I did for the portable C-wrapped waitpid > >? > >That way..the other code can do whatever it already is doing, >no need to come up with abstractions for looping with nohang, >vs. that plus the status loophole. > >The older fix I put it can be shrunk. > >The name of this is debatable, but I think the minimal level of abstraction is > goodness. > >The name is actually perhaps too..complicated..like, what does it mean? >Who uses it? What do they do with it? > >It means that waiting on a process to end allows threads in the waiting proces >s to run. >Perhaps it is too low level. > > > - Jay > > > >---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Date: Wed, 31 Dec 2008 05:33:22 +0000 >> Subject: [M3devel] variations of waitpid..? >> >> >> variations of waitpid.. >> >> Hey, we already have, that I put in on my initiative, months ago: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix. >m3 calls Pause(). *) >> VAR >> result: int; >> statusM3: Uwaitpid.waitpid_status_t; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); >> IF result # 0 THEN EXIT END; >> Pause(Delay); >> END; >> 0 *> >> RETURN statusM3.w_Loophole; >> END WaitProcess; >> >> pthreads and cygwin: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 >> and >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) >> VAR >> statusM3: Uwaitpid.waitpid_status_t; >> BEGIN >> LOOP >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO >> IF r> 0 THEN >> RETURN statusM3.w_Loophole; >> END; >> < 0*> >> END; >> >> END; >> END WaitProcess; >> >> win32: >> PROCEDURE WaitProcess (pid: int): int = >> BEGIN >> >> RETURN 0; >> END WaitProcess; >> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po >int. >> >> However, now, to fix sysutils, we need something LIKE: >> >> UNSAFE INTERFACE SystemPosixWaitPid; >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; >> >> BEGIN >> END SystemPosixWaitPid. >> >> pthreads: >> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> BEGIN >> RETURN waitpid(pid, status, 0); >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidEfficient. >> >> user threads: >> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> VAR result: int; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); >> IF result # 0 THEN >> EXIT >> END; >> Thread.Pause(Delay) >> END; >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidPause. >> >> questions: >> >> Aside: Where do I put "unsafe"? >> >> Relevant: Sysutils has to implement this itself, right? >> Due to bootstrapping concerns. >> >> But we should still put /something/ in m3core, right? >> >> So then the big question, what to call it? >> >> Probably, the existing WaitProcess that does more than this WaitPid, should >be scaled >> back and only do what WaitPid does? >> >> I will likely commit exactly what is shown above. >> The new interface and modules are in sysutils. The interface is private. >> >> The unfinished part is putting something in m3core, that sysutils could use, > if >> not for boostrapping issuers, that it could use in the future. >> >> I kind of thing something a bit more is needed. >> You know, the Win32 implementation that asserts(false) is suspicious. >> At the very least, it should be removed. >> >> I suspect I am missing something though, as suggested by Win32 having an ass >ert(false) implementation. >> Either that needs to be removed, or somehow the code can be unified, using j >ust the existing m3core/libm3 >> interfaces? >> >> I also suspect part of the problem is that sysutils implements to a large ex >tent >> the same abstractions as m3core and/or libm3, but with variations, causing i >t to be unable >> to build on what is there, but having to reinvent..which suggests the m3core >/libm3 abstractions >> maybe are not general/featureful enough. >> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth >reads.) >> >> >> ?? >> >> - Jay From mika at async.caltech.edu Wed Dec 31 09:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 31 Dec 2008 00:22:41 -0800 Subject: [M3devel] TEXT & etc. Message-ID: <200812310822.mBV8Mf2j027994@camembert.async.caltech.edu> Hello everyone, I mentioned a week or two ago that I had run into horrific performance issues with CM3's TEXTs. It wasn't my intention to start a debate about Unicode and Mahjong characters, just to point out that there's a serious performance problem with CM3 in this area. I run into performance problems with CM3 from time to time because I am trying to maintain a fairly large amount of software written in Modula-3 and it appears to me that the SRC/PM3 compilers often pay better attention to performance issues than the newer CM3 compiler. Generally speaking I think Modula-3 needs to avoid getting into the Java situation: Java is squeezed between "fast enough" Python and "much more efficient" C++... Modula-3 I see as the only hope for getting the performance of C++ with the safety of Java. The problems I have seen with CM3 are in the following areas: 1. Thread mutex acquisition involving kernel calls (pthreads implementation) 2. ISTYPE and TYPECASE (appear much faster on PM3, don't know why, could just be a profiling issue). In any case TYPECASE is very slow because of having to search the type tree. This could be improved but I can't work on it because I always run into some kind of fatal problem trying to bootstrap the compiler. 3. The TEXT issue. I mentioned performance problems before. It turns out that it was showing up in another area. I upgraded my PowerBook a while back (to OS X 10.4) and of course everything stopped working, including at least half my Modula-3 programs. I thought it was a versioning problem with C libraries I was importing, but it turned out that in at least one case I was running out of stack space because of the calls to get_char on concatenated TEXTs. (Not very long ones, at that.) To get to the point. What follows is my new version of TextCat.m3, which I believe works, but I can't really say because I'm perennially incapable of bootstrapping CM3. The basic idea is that you can have *one* TextCat.T, at the "root" of your TEXT. It will flatten anything else. I don't see a simple way of modifying the CM3 TEXT implementation to do much better than this. Of course throwing out TextCat.T completely is an option. An advantage of what I've done is that I haven't changed any interfaces at all. So I don't get problems with version stamps, etc. Mika (* Copyright 1996-2000, Critical Mass, Inc. All rights reserved. *) (* See file COPYRIGHT-CMASS for details. *) UNSAFE MODULE TextCat EXPORTS RTHooks, TextCat; IMPORT TextClass, Text8, Text16; (* changed 12/31/2008: only return a T when neither of the arguments to Concat is already a T; otherwise flatten to a Text8 or Text16. Change affects only Concat and MultiCat *) REVEAL T = Public BRANDED "TextCat.T" OBJECT OVERRIDES get_info := MyGetInfo; get_char := MyGetChar; get_wide_char := MyGetWideChar; get_chars := MyGetChars; get_wide_chars := MyGetWideChars; END; PROCEDURE New (t, u: TEXT): TEXT = BEGIN RETURN Concat (t, u); END New; (* RTHooks.Concat -- called by the inline "&" operator *) PROCEDURE Concat (t, u: TEXT): TEXT = VAR ti, ui: TextClass.Info; BEGIN t.get_info (ti); IF (ti.length <= 0) THEN RETURN u; END; u.get_info (ui); IF (ui.length <= 0) THEN RETURN t; END; IF NOT ISTYPE(t,T) AND NOT ISTYPE(u,T) THEN RETURN NEW (T, a := t, b := u, a_len := ti.length, b_len := ui.length, a_or_b_wide := ti.wide OR ui.wide) ELSIF ti.wide OR ui.wide THEN WITH res = Text16.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_wide_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_wide_char(i) END; RETURN res END ELSE WITH res = Text8.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_char(i) END; RETURN res END END END Concat; PROCEDURE NewMulti (READONLY x: ARRAY OF TEXT): TEXT = BEGIN RETURN MultiCat (x); END NewMulti; (* RTHooks.MultiCat *) PROCEDURE MultiCat (READONLY x: ARRAY OF TEXT): TEXT = VAR wide := FALSE; length, start := 0; xi : TextClass.Info; BEGIN IF NUMBER (x) <= 0 THEN RETURN ""; END; IF NUMBER (x) = 1 THEN RETURN x[0]; END; IF NUMBER (x) = 2 THEN RETURN Concat(x[0],x[1]) END; FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); wide := wide OR xi.wide; length := length + xi.length END; IF wide THEN WITH res = Text16.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_wide_char(j) END; start := start + xi.length END; RETURN res END ELSE WITH res = Text8.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_char(j) END; start := start + xi.length END; RETURN res END END END MultiCat; PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; PROCEDURE MyGetChar (t: T; index: CARDINAL): CHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetChar; PROCEDURE MyGetWideChar (t: T; index: CARDINAL): WIDECHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_wide_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_wide_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetWideChar; PROCEDURE MyGetChars (t: T; VAR a: ARRAY OF CHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetChars; PROCEDURE MyGetWideChars (t: T; VAR a: ARRAY OF WIDECHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetWideChars; BEGIN END TextCat. From roland.illig at gmx.de Wed Dec 31 11:33:31 2008 From: roland.illig at gmx.de (Roland Illig) Date: Wed, 31 Dec 2008 11:33:31 +0100 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? Message-ID: <495B4A7B.6010505@gmx.de> Hi, when trying to build the simplest program, examples/hello, I get the following error messages: $ ~/usr.local/packages/cm3-5.4.0/bin/cm3 --- building in LINUXLIBC6 --- new source -> compiling Hello.m3 Hello.ms: Assembler messages: Hello.ms:21: Error: suffix or operands invalid for `push' Hello.ms:23: Error: suffix or operands invalid for `push' Hello.ms:27: Error: suffix or operands invalid for `pop' ... Is there some way of running cm3 on an x86_64 system? Roland From rcoleburn at scires.com Wed Dec 31 15:49:32 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 09:49:32 -0500 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <495B3F1E.1E75.00D7.1@scires.com> I agree with Tony about use of Modula-3 vs. C. Modula-3 is a systems programming language. We should be using it for systems programming, not C. My vote is to always use Modula-3, not C. Regards, Randy >>> Tony Hosking 12/30/2008 6:47 PM >>> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:28:37 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:28:37 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> Message-ID: <495B4846.1E75.00D7.1@scires.com> Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:46:27 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:46:27 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> References: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> Message-ID: <495B4C74.1E75.00D7.1@scires.com> Jay: The "cm3-41-hack.c" file is an old file that isn't used anymore. I seem to recall that the CMass folks created it for me back in the days of v4.1 to get around a bug. When I committed the cm3ide package a while back, I also committed the WindowsResoures package. I kept the cm3-41-hack.c file in there for historical purposes just in case someone ever has occasion to run cm3 v4.1. Indeed, anyone who purchased the CMass "Reactor" product (as I did) has v4.1 of cm3. You will note in the WindowsResources m3makefile that the cm3-41-hack.c file is commented out and there is a long comment proceeding it to explain its purpose. On a cm3 v4.1 system, you would either need to uncomment this line, or fix the underlying bug that prompted the need for it in the first place. Regards, Randy >>> Jay Krell 12/31/2008 11:03 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:03:11 Modified files: cm3/m3-sys/windowsResources/src/: cm3-41-hack.c Log message: microoptimize and reformat -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:58:44 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:58:44 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105714.24CF710D5DDE@birch.elegosoft.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> Message-ID: <495B4F55.1E75.00D7.1@scires.com> Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 18:59:03 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 12:59:03 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: <495B6B87.1E75.00D7.1@scires.com> Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy >>> 1/1/2009 5:25 AM >>> On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote: > > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation. It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine? > My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad. Except that the free download won't work on old versions of Windows. This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available. > I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents. > > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system. > > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own. It is not hard to write a linker in Modula-3. > From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment). What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice. > > Regards, > Randy -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:34:24 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:34:24 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:44:00 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:44:00 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4F55.1E75.00D7.1@scires.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy>>> Jay Krell 12/31/2008 11:57 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:57:14Added files:cm3/m3-sys/windowsResources/src/: m3overrides Log message:Add missing override to fix some builds.I'm highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:54:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:54:53 +0000 Subject: [M3devel] FW: [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: truncated, around the cmd/bat stuff From: jay.krell at cornell.eduTo: rcoleburn at scires.com; m3devel at elegosoft.comSubject: RE: [M3devel] [M3commit] CVS Update: cm3Date: Wed, 31 Dec 2008 18:34:24 +0000 > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view.They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT.They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns.Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided.They are truly identical in every respect in the front end and back end.They only vary in the config file.This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have.Therefore, BUILD_DIR is arbitrary.I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so..I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't.Show me where it does.The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific.I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe).I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH.True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python.cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB.That is also a competing workaround for paths with spaces.And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does).Personally I find cmd/bat among the worst programming languages ever and would rather not write it.jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows.That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it.Python is a very decent programming language that is very portable across "all" systems.Perl would be the next best choice, though..I just don't like much.sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win.I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself.It shouldn't be all that hard. Modula-3 still needs a C linker.And there is C code that I didn't put in -- hand.c and dtoa.c.If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning".You pretty much must chose one.The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:51:42 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:51:42 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: > It is not hard to write a linker in Modula-3. By all means then, do it.> What's hard is making it compatible with existing proprietary linkers > and loaders that are poorly documented and subject to change without > notice. Good point -- do you want to be able to debug with Visual Studio or cdb/ntsd/windbg? You are in for more work then. I don't think it as ever-changing as claimed, but it isn't easy either. For Windows, you should be able to write the symbols to the .exe or a separate .dbg file, rather than deal with the truly undocumented and changing .pdb file format (there is a third party book that mostly documents it). Do the other systems change a lot? If you look across a wide range of time, well, then, yeah, you get to handle the various COFF to ELF transitions, and Apple's rewrite every few years (68K => XCOFF/CFM/PEF => MachO). They are also largely unavoidably system specific, so writing it just once won't suffice. You know..you have to learn about and handle object file formats, at least three of them, and executable file formats, again at least three -- ELF, (MS/PE)COFF, and Mach-O. And then for debugging, there is CodeView, stabs, DWARF, and maybe others. If you only want gdb, then you can just stick with stabs. (Other options seem to crash cm3cg -- like when I fiddle around with -g vs. -ggdb vs. -gstabs.) Or you can just debug with printf and have no symbols...ugh... - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:05:26 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:05:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B6B87.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> <495B6B87.1E75.00D7.1@scires.com> Message-ID: > All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary > distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I am curious if anyone wants it too, but it is not difficult. Just run the vcvars32, set USE_DELAYLOAD=0 for versions prior to roughly 6.0 or 5.0spsomething (Modula-3-ism, and if anyone really wanted, could just remove it and back it just work always), and run the existing automation. It should just work. You don't have to be running on NT4, there isn't the global /usr/include business as on Unix. The headers and libraries are part of tools and not part of the operating system. Even using the latest toolsets might just work. The compiler and linker continue to output very generic stuff. The main area that platform dependencies creep in is the C runtime. For example, IsDebuggerPresent() is not on NT3.51 or Win95, but is used in newer C runtime. Some of the Interlocked functions as well, but you can also just use intrinsics and inlines for them, and then worry about the procesor and not the operating system. e.g. some of the Interlocked intrinsics/inlines require a 486, and some require something even newer -- the InterlockedCompareExchange64 stuff, if we have any. Even so, that's probably on all processors under 10 years old. XP requires it I think, but NT4/Windows 2000/Win9x did not. I think 486 is required for the modern variants of InterlockedIncrement/Decrement -- which return the entire new value, and not merely <0, 0, >0. InterlockedSomething is actually a small issue. The current C runtimes use InterlockedSomething that aren't exported on older (like pre-Win2000) systems. InterlockedIncrement/Decrement..kinda scary..you know, the functions changed meaning, in a compatible way, just from a binary's imports, you can't tell if it requires the newer meaning or if the older meaning suffices. To be clear, the newer meaning satisfies the older contract, but the older meaning doesn't satisfy the newer contract. Anyway, the modern way is to inline/intrinsic them and require a 486. But Modula-3's use of the C runtime is very small. And I dare say..we should try to remove it entirely... - Jay Date: Wed, 31 Dec 2008 12:59:03 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy>>> 1/1/2009 5:25 AM >>>On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote:> > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation.It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine?> My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad.Except that the free download won't work on old versions of Windows.This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available.> I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents.> > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system.> > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own.It is not hard to write a linker in Modula-3.> From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment).What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice.> > Regards,> Randy-- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:14:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:14:02 +0000 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? In-Reply-To: <495B4A7B.6010505@gmx.de> References: <495B4A7B.6010505@gmx.de> Message-ID: Mostly yes. There is not always a full assortment of runtime libraries or development/static libs, like on a minimal Debian system. In your config file, make sure to put -m32 on the cm3cg invocation, and perhaps -32 or --32 on the as/gas invocation. They may or may not help. Or hurt. (They can hurt in that older builds don't understand them and error right way. They can help if you have biarch tools that default to 64bit. They make no different on modern tools that are only 32bit or biarch tools that default to 32bit). Or just try config files. Though I don't test this out much. I can look into it soon..late next week at the latest. > Hello.ms:21: Error: suffix or operands invalid for `push' Can you show us Hello.ms line 21, or the entire Hello.ms file? There is also a "native" AMD64_LINUX port that you might try. "biarchness" is nice and convenient, to kind of get two systems in one, but it can also be confusing and easily misused. OpenBSD is interesting here -- never any biarchness, simpler. - Jay> Date: Wed, 31 Dec 2008 11:33:31 +0100> From: roland.illig at gmx.de> To: m3devel at elegosoft.com> Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu?> > Hi,> > when trying to build the simplest program, examples/hello, I get the> following error messages:> > $ ~/usr.local/packages/cm3-5.4.0/bin/cm3> --- building in LINUXLIBC6 ---> > new source -> compiling Hello.m3> Hello.ms: Assembler messages:> Hello.ms:21: Error: suffix or operands invalid for `push'> Hello.ms:23: Error: suffix or operands invalid for `push'> Hello.ms:27: Error: suffix or operands invalid for `pop'> ...> > Is there some way of running cm3 on an x86_64 system?> > Roland> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:24:07 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:24:07 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> References: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> Message-ID: You mean, this function is easy to misuse? > People who declare their own <*EXTERNAL*> Like waitpid exposed from m3core? waitpid is already easy to misuse, on a userthread system, leading to possible (though I think rare) deadlock. It is easy to misuse on pthreads, lead "just" to bad performance, and in fact I believe cm3 is doing this, via sysutils. This at least guides you between two patterns of use, and fix the perf of cm3/sysutils. On a userthread system, waitpid(pid, flags = 0) waits for the child process, with all parent threads suspended. Generally I doubt the child depends on parent threads progressing, but, yeah, that could deadlock, like if a parent thread is waiting to a file or stdin of the child, or reading a child's stdout. The various uses do waitpid(pid, flags = nohang) and then sleep and try again. pthreads just uses waitpid(pid, flags = 0) and all threads keep running. Win32 just uses WaitForSingleObject(process handle, timeout=infinite), and again all threads keep running. Just another problem with user threads, where pthreads and Win32 are fine. > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? WaitPROCESS always yields. It is WaitPID that may or may not. So I renamed the function to DoesWaitPidYield() and moved it to SchedulerPosix.i3. ok? - Jay> To: jay.krell at cornell.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] variations of waitpid..? > Date: Tue, 30 Dec 2008 23:22:41 -0800> From: mika at async.caltech.edu> > > I don't think it should be possible to stop other threads if the> runtime could even conceivably support an environment where that's> not necessary. You're introducing dependencies that will lead to> (undetectable) deadlocks. People who declare their own <*EXTERNAL*>> things are, as usual, on their own.> > Mika> > Jay writes:> >> >Hm. How about:> > > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN?> > > >?> >true for pthreads/win32/cygwin> >false for userthreads> > > >duplicated privately in sysutils to avoid depending on newer m3core> > like I did for the portable C-wrapped waitpid> > > >?> > > >That way..the other code can do whatever it already is doing,> >no need to come up with abstractions for looping with nohang,> >vs. that plus the status loophole.> > > >The older fix I put it can be shrunk.> > > >The name of this is debatable, but I think the minimal level of abstraction is> > goodness.> > > >The name is actually perhaps too..complicated..like, what does it mean?> >Who uses it? What do they do with it?> > > >It means that waiting on a process to end allows threads in the waiting proces> >s to run.> >Perhaps it is too low level.> > > > > > - Jay> >> >> >> >----------------------------------------> >> From: jay.krell at cornell.edu> >> To: m3devel at elegosoft.com> >> Date: Wed, 31 Dec 2008 05:33:22 +0000> >> Subject: [M3devel] variations of waitpid..?> >>> >>> >> variations of waitpid..> >>> >> Hey, we already have, that I put in on my initiative, months ago:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.> >m3 calls Pause(). *)> >> VAR> >> result: int;> >> statusM3: Uwaitpid.waitpid_status_t;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG);> >> IF result # 0 THEN EXIT END;> >> Pause(Delay);> >> END;> >> 0 *>> >> RETURN statusM3.w_Loophole;> >> END WaitProcess;> >>> >> pthreads and cygwin:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3> >> and> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *)> >> VAR> >> statusM3: Uwaitpid.waitpid_status_t;> >> BEGIN> >> LOOP> >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO> >> IF r> 0 THEN> >> RETURN statusM3.w_Loophole;> >> END;> >> < 0*>> >> END;> >>> >> END;> >> END WaitProcess;> >>> >> win32:> >> PROCEDURE WaitProcess (pid: int): int => >> BEGIN> >>> >> RETURN 0;> >> END WaitProcess;> >>> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po> >int.> >>> >> However, now, to fix sysutils, we need something LIKE:> >>> >> UNSAFE INTERFACE SystemPosixWaitPid;> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int;> >>> >> BEGIN> >> END SystemPosixWaitPid.> >>> >> pthreads:> >>> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> BEGIN> >> RETURN waitpid(pid, status, 0);> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidEfficient.> >>> >> user threads:> >>> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> VAR result: int;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG);> >> IF result # 0 THEN> >> EXIT> >> END;> >> Thread.Pause(Delay)> >> END;> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidPause.> >>> >> questions:> >>> >> Aside: Where do I put "unsafe"?> >>> >> Relevant: Sysutils has to implement this itself, right?> >> Due to bootstrapping concerns.> >>> >> But we should still put /something/ in m3core, right?> >>> >> So then the big question, what to call it?> >>> >> Probably, the existing WaitProcess that does more than this WaitPid, should > >be scaled> >> back and only do what WaitPid does?> >>> >> I will likely commit exactly what is shown above.> >> The new interface and modules are in sysutils. The interface is private.> >>> >> The unfinished part is putting something in m3core, that sysutils could use,> > if> >> not for boostrapping issuers, that it could use in the future.> >>> >> I kind of thing something a bit more is needed.> >> You know, the Win32 implementation that asserts(false) is suspicious.> >> At the very least, it should be removed.> >>> >> I suspect I am missing something though, as suggested by Win32 having an ass> >ert(false) implementation.> >> Either that needs to be removed, or somehow the code can be unified, using j> >ust the existing m3core/libm3> >> interfaces?> >>> >> I also suspect part of the problem is that sysutils implements to a large ex> >tent> >> the same abstractions as m3core and/or libm3, but with variations, causing i> >t to be unable> >> to build on what is there, but having to reinvent..which suggests the m3core> >/libm3 abstractions> >> maybe are not general/featureful enough.> >>> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of> > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth> >reads.)> >>> >>> >> ??> >>> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:30:40 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:30:40 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: <495B901F.1E75.00D7.1@scires.com> Jay: First, I do want to say thanks for all you are doing for the cause of Modula-3. I don't think we say thanks enough for what others do for us. THANK YOU! I don't want to be perceived as "complaining", rather I am trying to voice my opinion in the hopes of influencing future direction. Of course, since I'm not doing the work, I can only make suggestions. In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. Now, I'm not sure how much effect this deoptimization has (maybe little), but it is clear that in this case your implementation choice has favored the non-native implementation over the native one. IMO, tradeoffs of this type are not good. If one is trying to convince someone to use Modula-3, why would you want to give them a "deoptimized" version just to make it easier to support a non-native environment---indeed, one that they may not even want/use? If we have to make a trade-off, I say favor the native implementation always over the non-native one. What I'm trying to say here is that my experience as a software, systems, and program engineer is that I've always been forced to justify the cost/benefit of development tools for any project. Many times I've had to go head to head with folks higher up in my own organization or in the customer's organization whose preconceived opinions were based on rumor and what they've heard rather than actual factual hands-on experience. I want to pick the best tool for the job instead of being forced to pick an inferior tool because someone higher in the food chain demanded it based on faulty preconceived opinion. I like Modula-3. I've found that I am more productive using it. But, if the compiler doesn't produce efficient code, I lose ground in arguing with the higher-ups. As for Python, I've never ventured to learn it, so for me, it is something of a mystery. But you miss the point. I'm not arguing the merits of Python, I'm just saying that anything Modula-3 requires on top of what is provided by the standard host platform represents a potential obstacle or barrier to ease of use/implementation. It also sends the message that somehow Modula-3 is not complete on its own, we have get Python just to install and oh yes we need a C compiler and a linker and a ...? IMO, ultimately we need a turn-key download and install routine for Modula-3 that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install, I can run it on Windows, but if you give me an install routine that requires I first install something else, e.g. Python, then that becomes a barrier to the folks who don't know Python or already have it installed. Am I alone in this line of thought? If so, I'll just be quiet. BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. My problem is that right now, it is hard to understand all the dependencies for proper building of cm3 from scratch. Indeed, I've contributed some CMD/BAT install stuff in the past, but it is now out of date. Perhaps if there is a way we could better document the proper build order and dependencies, it would be easier for folks in the community to help in this area. I certainly would volunteer to do the CMD files as long as I had enough information to do it right. At one point, I thought we had a file that showed the package build order and dependencies. Not sure if the format of this file is well-documented or if the file is kept up-to-date. Regards, Randy >>> Jay 12/31/2008 1:34 PM >>> > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:39:54 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:39:54 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: <495B9248.1E75.00D7.1@scires.com> Jay: I was just trying to explain a bit about the history of and reason for the windowsResources package. As for m3overrides, I rarely use it anymore. At one point in time, we had a large project going on and the engineers used m3overrides for local testing before shipping to the main repository. That way, we avoided committing something that was obviously broken and causing other engineers to have downtime. I also seem to recall devising a neat way to use m3overrides to build against stubs and to create test editions of a module. Alas, it's been a long time since I've managed a project with many Modula-3 engineers working concurrently. Regards, Randy >>> Jay 12/31/2008 1:44 PM >>> "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 21:55:13 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 20:55:13 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B901F.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <495B901F.1E75.00D7.1@scires.com> Message-ID: > In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. It is such a slight deoptimization, I think it is ok. Are you comfortable with only slightly optimizing never inlining integrated backend? Not optimal. But it sure does build faster than gcc. Great for productivity. And its "always only slightly optimizing" mode generates much better code than gcc's default "no optimization at all" mode. But gcc with optimizing should be able to beat it handily -- e.g. inline. Do you use gcc's optimizer or not? I usually don't. I want to be able to debug line by line with a close correlation of binary to source. I have plenty of experience debugging optimized code, and it can be an interesting fun educational challenge, but unless I'm really bored, I'd rather not do it. If you don't use the optimizer, then again, not optimal. I wouldn't mind /some/ optimization, esp. some inlining and constant propagation, but I dislike it when the debugger jumps around with no clue as to what "line" of code I am on (because "lines" of code often don't exist in optimized code, a line can go away, or get split up, or become shared with other "lines", etc.). Are you aware of the cost of PushEFrame/try/raise/finally in Modula-3? Nobody seems to care much. Granted, SOLgnu/sun does it better, so people care.Almost anything is more efficient. Windows/x86 C/C++ does something similar but way more optimized. All other C/C++ systems use "stack walking" and incur little cost from the "try" alone. Though raise is often more expensive. Ok, granted, my deoptimization is to make PushEFrame even less efficient. It is already space (stack) and CPU inefficient. I made it more space (stack) inefficient. Could contribute to stack exhaustion. > IMO, ultimately we need a turn-key download and install routine for Modula-3 > that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install Can you try the uploaded archives I produced? Are they really too difficult to use?You extract them, optionally rename the root directory, run vcvars, add cm3 to %PATH%, and are done. Arguably much better than the older interactive cminstall. Not sure about the non-interactive variant. I understand your point. No Python is needed to download and run. No Python is really needed to build the system, but cm3 itself only builds a package at a time. The sh and Python automation is around cd'ing around and building, in the right order, with some filtering, and my Python adds ability to pick target platform on the command line, without messing around every time with config files and %PATH% and such. It is nice and convenient, it adds features that cm3 doesn't provide, but you don't have to have it. > BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. Your cmd/bat are actually still probably correct or very nearly so. The dependency orders rarely/never change. The main change is that sysutils was added. And perhaps that the dependency order became "more important" but not actually "different". That is, probably for a long time, an old cm3 could compile a current m3core/libm3. Now you need a cm3 with LONGINT support to build them (which itself is now a few years old). So you must first build the compiler with the old compiler, THEN build the current runtime. The "upgrade" scripts handle this. Again you can just cd around in the right order, with some repetition (you build the compiler first against the old runtime, then build the current runtime, then rebuild the compiler against the new runtime; arguably not needed but very much what you want to do). [redundancy here due to bad editing, sorry] Also, if you are bootstrapping from an older build, then you use their m3core and libm3, and then build the compiler up in dependency order starting with sysutils. If you are building from scratch with a current compiler, then you can start with m3core and go up from there. (Or import-libs on Windows, if needed, but this really just a workaround for old distributions, like 4.1, like before Tony's change to remove the "gcdefs" stuff.) > At one point, I thought we had a file that showed the package build order and dependencies. I was a big proponent of this, and it now exists and is in use..but I don't use it usually. :) It is the scripts/pkginfo.txt file. All the sh scripts use it I believe. Therefore the tinderbox uses it I believe. My cmd that I don't use (scripts/win) I think use it. My python code does not use it. Oops. Well, I think I did implement it at least, so I didn't push someone else to and then not use it.. :) (Pick your evil -- do something yourself, and then not use it, or push someone else to, and then not use it). Eh, I should just use it from the Python, not a big deal. Not only the build order, but the file also aids in package set selection. It doesn't do the filtering though, oh well. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Dec 13 19:03:08 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 13 Dec 2008 18:03:08 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a);END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:08:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:08:58 +0000 Subject: [M3devel] code duplication.. Message-ID: Anyone want to combine these identical files into one shared one somewhere? m3-mail/postcard/src/OSUtils.m3m3-mail/webcard/src/OSUtils.m3 - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 06:53:36 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 05:53:36 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 14 07:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 14 Dec 2008 17:23:06 +1100 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> Message-ID: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: > I tried getting C to error this way, and failed. > It looks like C uses COMPONENT_REF here, which will send it down > different paths I didn't follow. > COMPONENT_REF makes sense for it and only somewhat sense for us. > That would involve declaring all of a module's globals a struct, > which I don't believe we currently do but might be good. I think > there's a basic disconnect in that in C, bitfields are always > fields, never freestanding. Modula-3 rather violates that. Seems a > little dangerous therefore to be using them this way? > > In terms of the inefficient option in parse.c, that would maybe go > away, as all the offseting would be handled by the next level down, > like, implementation of fields? > Well..that is assuming the only use offset != 0 in m3_load is for > accessing module globals. I don't know if that is true. > > Maybe another option, should be easy to implement but might compile > slowly, is to cons up a field decl right there in m3_load/store? > That would lead to a lot of extra nodes -- per access instead of > declared once. I can try that out. > > - Jay > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sat, 13 Dec 2008 18:03:08 +0000 > > I've just spent a few hours debugging this. > > The problem is strongly related to the declaration that the global > segment is of size BIGGEST_ALIGNMENT. > gcc decides they can fit in a register..so then it doesn't bother > adding the offset to the address, something like that. > > When things work, it goes here: > op0 = adjust_address (op0, mode1, offset); > in expr.c. When things don't work, that line is skipped and then > shortly below that: > > if (mode1 != mode) > return convert_to_mode (tmode, op0, unsignedp); > return op0; << > > On systems where BIGGEST_ALIGNMENT fits in a register, e.g. > apparently MIPS, counter-e.g. apparently many others, but not > clearly all others. This determines whether the mode of the segment > is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger > than any machine type?). BLKmode works. It works on 32bit MIPS > because between mode1 and mode, one of them is SImode, one is > DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since > biggest alignment is set very large -- 128. I don't know why it > works for SPARC. > > PERHAPS I can put together a C or treelang example and get more (gcc > developers) eyes on it, but I doubt it. > > The following hack works around it: > > C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c > Index: parse.c > =================================================================== > RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v > retrieving revision 1.77 > diff -r1.77 parse.c > 2865c2865 > < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, > BIGGEST_ALIGNMENT); > --- > > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, > BIGGEST_ALIGNMENT); > > The repro is very easy. > Just add this to m3core: > > MODULE Main2 EXPORTS Main; > IMPORT RTIO; > VAR a := "a"; > BEGIN > RTIO.PutText(a); > END Main2. > > The load of a will be from MM_Main2 offset 0, instead of 104 or > thereabouts. > > I'll commit this shortly. > > - Jay > > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > Date: Thu, 13 Nov 2008 00:04:32 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > Jay, do you have access to an AMD64_DARWIN box? > > No, sorry, I don't yet and not sure when I will. > Unless it is easy enough to run on non-Apple hardware. > I'll have to debug it (m3cg) then. > > - Jay > > > > > > > > CC: m3devel at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Wed, 12 Nov 2008 20:46:53 +0000 > > Jay, do you have access to an AMD64_DARWIN box? I did get things > working on that 64-bit platform. Or is it an endianness issue you > are dealing with. In any case, I will not have a lot of bandwidth > for doing much in the way of tackling this unless it becomes > critical path for me soon. > > > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 12 Nov 2008, at 07:57, Jay wrote: > > Target.Little_endian is used to drive some target-specificity in the > IR. > I guess, after all, that is why you have to specify it. > > The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and > SPARC64_SOLARIS, once I set them both to unaligned_procedures, which > they both need. (Tangent, I ran m3cgcat on the .ms file a few times > instead of the .mc file; that makes it crash.) > > I can't claim SPARC64_SOLARIS is any paragon of working and correct > though, by far. > I should also be able to compare with SPARC64_OPENBSD, I forgot > about that, and didn't have an m3cg. > I only ran SPARC64_SOLARIS far enough to discover it needed > unaligned procedures. > I think I should fork linux-common and solaris-common off of the > minimal 29k openbsd-common, rather than worry about all the stuff in > solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS > could/would be in better shape probably. > > MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along > with the parse.c fix -- it gets to a bus error in RunMainBody. > Again an alignment problem, though I think maybe memory memory > trashing. > The fault occurs here: > > desc_offset := init_depth * ADRSIZE (InitDesc); > > desc := init_stack + desc_offset; > init_depth := desc.low_link; << > > InitDesc = RECORD > module : RT0.ModulePtr; > low_link : INTEGER; > END; > > desc_offset appears to not be a multiple of 4. > > I will look into that. > > Long story short -- consider again the parse.c change? > I admit I still haven't debugged it (or rather, the code that > consumes the data produced by it). > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Date: Sun, 9 Nov 2008 11:10:54 +0000 > > I agree. I am suspicious too. I will investigate more. Thanks. > > - Jay > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sun, 9 Nov 2008 10:01:58 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > > > Aren't the integer values supposed to be written endian- > independently in the IR? I am suspicious about these differences. > > On 9 Nov 2008, at 01:25, Jay wrote: > > Thanks for the tip/direction. > > Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the > same" but for two systematic diffs: > > - AMD64_LINUX has a bunch of added calls to check_nil. > This is a "bug" in it, inoptimal but ok code, because "first > readable address" is 0. It should probably be 4k. I should write a > program that iterates starting at 0 to find the value. > (or better yet, cm3 should have a -configure option that does > this? -- for configuration values for which there is a safe default; > in general, imho, the targets should be auto configured via C, > assuming there is a native development environment available). > > - some integers are different (again, systematically) > > AMD64_LINUX: > init_int 656 13 Int.64 > init_chars 664 " typecode " > init_int 680 2 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 2 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 2 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 2 Int.64 > init_var 816 v.1 0 > init_int 824 17 Int.64 > init_chars 832 "ExpandInitStack: " > > MIPS64_OPENBSD: > > init_chars 664 " typecode " > init_int 680 8796093022208 Int.64 > init_var 688 v.1 0 > init_int 696 13 Int.64 > init_chars 704 " typeid " > init_int 720 8796093022208 Int.64 > init_var 728 v.1 0 > init_int 736 13 Int.64 > init_chars 744 "RunMainBody: " > init_int 760 8796093022208 Int.64 > init_var 768 v.1 0 > init_int 776 19 Int.64 > init_chars 784 "RunMainBody: exec: " > init_int 808 8796093022208 Int.64 > init_var 816 v.1 0 > > This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where > little endian has 2. > > I don't know what these numbers represent. > I should look at other systems -- preferably a working 64 bit big > endian system, if there are any. > > Could be flags/type with text literals? > > (Btw, I always leave the TextLiteral.i3 change on my system, > lowering the maximum literal length to around 4gig even on 64 bit > targets, so 32 bit can bootstrap 64 bit...need to fix the compiler > here to do target arithmetic instead of host arithmetic...) > > Thanks, > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Sat, 8 Nov 2008 22:55:54 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > > Have you checked to make sure that the pre-gcc IR makes sense? Does > it see these values as 32-bit or 64-bit? The reason I ask is that > there may be something wrong in your m3middle target specs that > cause bad IR. Once we know the IR looks right then we can try and > fix the problem. > > Antony Hosking | Associate Professor | Computer Science | Purdue > University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > On 8 Nov 2008, at 15:45, Jay wrote: > > > truncated as usual...along with losing a lot of whitespace.. > Let's see... > > > Tony, how about switching these #if's? > I explain some of why below. > > static void > m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > tree val; > #if 0 > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != dst_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, dst_t, > m3_cast (build_pointer_type (dst_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > val = m3_cast (src_t, EXPR_REF (-1)); > if (src_T != dst_T) { > val = m3_build1 (CONVERT_EXPR, dst_t, val); > } > add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); > EXPR_POP (); > } > m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, > m3_type dst_T) > { > #if 0 > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o)); > } > #else > /* failsafe, but inefficient */ > if (o != 0 || TREE_TYPE (v) != src_t) { > v = m3_build1 (ADDR_EXPR, t_addr, v); > v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); > v = m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), v)); > } > #endif > TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing > problems */ > if (src_T != dst_T) { > v = m3_build1 (CONVERT_EXPR, dst_t, v); > } > EXPR_PUSH (v); > } > > I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). > In an early version it seems that many loads/stores of global > variables is wrong. > The "first" code affected is RTLinker_ExpandModuleTable. > > PROCEDURE ExpandModuleTable () = > CONST InitialTableSize = 500; > VAR new_mods: ADDRESS; n_bytes: INTEGER; > BEGIN > IF (modules = NIL) THEN > (* first time... *) > max_modules := InitialTableSize; > modules := Cstdlib.malloc (InitialTableSize * BYTESIZE > (RT0.ModulePtr)); > IF (modules = NIL) THEN Cstdlib.abort (); END; > ELSE > n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); > new_mods := Cstdlib.malloc (n_bytes + n_bytes); > IF (new_mods = NIL) THEN Cstdlib.abort (); END; > EVAL Cstring.memcpy (new_mods, modules, n_bytes); > Cstdlib.free (modules); > modules := new_mods; > INC (max_modules, max_modules); > END; > END ExpandModuleTable; > > It should read back modules as NIL. > What happens is it reads back modules as not NIL, > and then max_modules should also be zero, but is "large", leading to > malloc to fail, > and abort called. > > The output of cm3cg -y is reasonable. > It indicates offset loads. > > What is incorrect is that in many loads and stores, but not all, the > offsets are lost. > > (923) begin_procedure > procedure RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable(924) set_source_line > source line 208 > (925) load_nil > (926) store > store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb > (927) set_source_line > source line 207 > (928) set_source_line > source line 210 > (929) load_nil > (930) load > m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb > > Here is the incorrect code: > > .globl RTLinker__ExpandModuleTable > .stabd 46,0,0 > .stabn 68,0,206,.LM108 > .LM108: > .LFBB8: > .set nomips16 > .ent RTLinker__ExpandModuleTable > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $3,MM_RTLinker > ld $2,128($3) > andi $2,$2,0x0 > ori $2,$2,0x1f4 > sd $2,128($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > ld $4,0($fp) > dla $3,MM_RTLinker > ld $2,136($3) > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK, offset seems low, but it agrees with > elsewhere > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > .stabn 68,0,216,.LM114 > .LM114: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > dsll $2,$2,3 > sd $2,8($fp) > .stabn 68,0,217,.LM115 > .LM115: > ld $3,8($fp) > ld $2,8($fp) > daddu $2,$3,$2 > move $4,$2 > jal malloc > sd $2,0($fp) > ld $2,0($fp) > sd $2,16($fp) > .stabn 68,0,218,.LM116 > .LM116: > ld $2,16($fp) > bne $2,$0,.L64 > jal abort > .L64: > .stabn 68,0,219,.LM117 > .LM117: > ld $4,16($fp) > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > ld $3,8($fp) > move $5,$2 > move $6,$3 > jal memcpy > sd $2,0($fp) > .stabn 68,0,220,.LM118 > .LM118: > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > move $4,$2 > jal free > .stabn 68,0,221,.LM119 > .LM119: > ld $4,16($fp) > dla $3,MM_RTLinker > ld $2,136($3) << OK > andi $2,$2,0x0 > or $2,$4,$2 > sd $2,136($3) << OK > .stabn 68,0,222,.LM120 > .LM120: > dla $2,MM_RTLinker > ld $3,0($2) << WRONG > dla $2,MM_RTLinker > ld $2,0($2) << WRONG > daddu $4,$3,$2 > dla $3,MM_RTLinker > ld $2,128($3) << OK > andi $2,$2,0x0 > or $2,$2,$4 > sd $2,128($3) << OK > .L65: > .LBE9: > .stabn 68,0,224,.LM121 > .LM121: > move $sp,$fp > ld $31,56($sp) > ld $fp,48($sp) > ld $28,40($sp) > daddiu $sp,$sp,64 > j $31 > .end RTLinker__ExpandModuleTable > > Here is corrected code: > > RTLinker__ExpandModuleTable: > .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 > .mask 0xd0000000,-8 > .fmask 0x00000000,0 > daddiu $sp,$sp,-64 > sd $31,56($sp) > sd $fp,48($sp) > sd $28,40($sp) > move $fp,$sp > lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > daddu $28,$28,$25 > daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) > .LBB9: > .stabn 68,0,208,.LM109 > .LM109: > sd $0,16($fp) > .stabn 68,0,210,.LM110 > .LM110: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT (offset seems low, but at least it > isn't zero) > ld $2,0($2) > bne $2,$0,.L61 > .stabn 68,0,212,.LM111 > .LM111: > dla $2,MM_RTLinker > daddiu $3,$2,128 << CORRECT > li $2,500 # 0x1f4 > sd $2,0($3) > .stabn 68,0,213,.LM112 > .LM112: > li $4,4000 # 0xfa0 > jal malloc > sd $2,0($fp) > dla $2,MM_RTLinker > daddiu $3,$2,136 << CORRECT > ld $2,0($fp) > sd $2,0($3) > .stabn 68,0,214,.LM113 > .LM113: > dla $2,MM_RTLinker > daddiu $2,$2,136 << CORRECT > ld $2,0($2) > bne $2,$0,.L65 > jal abort > b .L65 > .L61: > > I still dump core with this change, but I get much further. > I have not tested this change at all otherwise -- i.e. on working > platforms. > > I have not debugged through the gcc/m3cg code to see why > the offsets are only sometimes lost, or why other platforms > have no problem. Is MIPS's "global pointer" unusual these days? > I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms > had similar, but that's essentially nothing now (I'm not counting > IA64). > I know Linux has PLT (procedure linkage table) and GOT (global > offset table). > Same thing? > I haven't looked much at other MIPS ports yet. > > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 > > is what made it look about like it does -- in particularing using > BIT_FIELD_REF. > > 1.7: > > static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > m3_type src_T, dest_T; > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > tree adr = m3_build1 (ADDR_EXPR, t_addr, v); > if (o != 0) { > adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / > BITS_PER_UNIT)); > } > EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, > m3_cast (build_pointer_type (src_t), adr))); > } > #if 1 > if (src_T != dest_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); > } > if (debug_vars) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dest_T); > } > #else > if (src_T != dest_T) { > if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); > } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); > } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { > /* okay */ > } else { > fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", > src_T, dest_T); > } > } > #endif > } > static void > m3_store (v, o, src_t, dest_t) /* GCC32OK */ > tree v; > int o; > tree src_t, dest_t; > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dest_t) { > lhs = v; > } else { > tree f = make_node (FIELD_DECL); > TREE_TYPE (f) = dest_t; > DECL_ALIGN (f) = TYPE_ALIGN (dest_t); > DECL_SIZE (f) = TYPE_SIZE (dest_t); > DECL_MODE (f) = TYPE_MODE (dest_t); > DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); > DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); > DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); > lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); > } > #if 1 > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > #else > if (dest_t == NULL) { > expand_assignment (lhs, rhs, 0, 0); > } else { > expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, > 0); > } > #endif > EXPR_POP (); > } > > 1.8: > static void > m3_load (tree v, int o, > tree src_t, m3_type src_T, > tree dst_t, m3_type dst_T) /* GCC32OK */ > { > if (o == 0 && TREE_TYPE (v) == src_t) { > EXPR_PUSH (v); > } else { > EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), > bitsize_int (o))); > } > if (src_T != dst_T) { > EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); > } > if (option_vars_trace) { > const char *name = "noname"; > if (v != 0 && DECL_NAME(v) != 0) { > name = IDENTIFIER_POINTER(DECL_NAME(v)); > } > fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", > name, > o, src_T, dst_T); > } > } > static void > m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ > { > tree lhs, rhs; > if (TREE_TYPE (EXPR_REF (-1)) == src_t) { > rhs = EXPR_REF (-1); > } else { > rhs = m3_cast (src_t, EXPR_REF (-1)); > } > if (o == 0 && TREE_TYPE (v) == dst_t) { > lhs = v; > } else { > lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), > bitsize_int (o)); > } > if (src_t != dst_t) { > rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); > } > expand_assignment (lhs, rhs, 0, 0); > EXPR_POP (); > } > > - Jay > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:28:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:28:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct for size=align=maxalign, and then we access bit fields an offset from it. For if max is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 09:40:05 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 08:40:05 +0000 Subject: [M3devel] FW: m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: (truncated and reedited) Well, they are, definitely, but not with any fields, I think.Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it.If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang".Hopefully, going from just its name, it lets you build the exact trees that parse.c builds.As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supportedin current gcc without small patches (which cm3 applies).And then ask the gcc folks (cc you/all?)I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 14 Dec 2008 17:23:06 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed.It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow.COMPONENT_REF makes sense for it and only somewhat sense for us.That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields?Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sat, 13 Dec 2008 18:03:08 +0000I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT.gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.cIndex: parse.c===================================================================RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,vretrieving revision 1.77diff -r1.77 parse.c2865c2865< = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT);---> = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy.Just add this to m3core: MODULE Main2 EXPORTS Main;IMPORT RTIO;VAR a := "a";BEGIN RTIO.PutText(a);END Main2.The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduDate: Thu, 13 Nov 2008 00:04:32 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will.Unless it is easy enough to run on non-Apple hardware.I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Wed, 12 Nov 2008 20:46:53 +0000Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far.I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg.I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures.I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody.Again an alignment problem, though I think maybe memory memory trashing.The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END;desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change?I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Date: Sun, 9 Nov 2008 11:10:54 +0000I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sun, 9 Nov 2008 10:01:58 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: "MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent.I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Sat, 8 Nov 2008 22:55:54 +0000CC: m3devel at elegosoft.comSubject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset??Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace..Let's see...Tony, how about switching these #if's?I explain some of why below.static voidm3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){ tree val;#if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP ();}m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T){#if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); }#else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); }#endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v);}I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2).In an early version it seems that many loads/stores of global variables is wrong.The "first" code affected is RTLinker_ExpandModuleTable.PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable;It should read back modules as NIL.What happens is it reads back modules as not NIL,and then max_modules should also be zero, but is "large", leading to malloc to fail,and abort called.The output of cm3cg -y is reasonable.It indicates offset loads.What is incorrect is that in many loads and stores, but not all, theoffsets are lost.(923) begin_procedure procedure RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable(924) set_source_line source line 208(925) load_nil(926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb(927) set_source_line source line 207(928) set_source_line source line 210(929) load_nil(930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xbHere is the incorrect code:.globl RTLinker__ExpandModuleTable.stabd 46,0,0.stabn 68,0,206,.LM108.LM108:.LFBB8:.set nomips16.ent RTLinker__ExpandModuleTableRTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerld $2,0($2) << WRONGbne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $3,MM_RTLinkerld $2,128($3)andi $2,$2,0x0ori $2,$2,0x1f4sd $2,128($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)ld $4,0($fp)dla $3,MM_RTLinkerld $2,136($3)andi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerld $2,0($2) << WRONG bne $2,$0,.L65jal abortb .L65.L61:.stabn 68,0,216,.LM114.LM114:dla $2,MM_RTLinkerld $2,0($2) << WRONG dsll $2,$2,3sd $2,8($fp).stabn 68,0,217,.LM115.LM115:ld $3,8($fp)ld $2,8($fp)daddu $2,$3,$2move $4,$2jal mallocsd $2,0($fp)ld $2,0($fp)sd $2,16($fp).stabn 68,0,218,.LM116.LM116:ld $2,16($fp)bne $2,$0,.L64jal abort.L64:.stabn 68,0,219,.LM117.LM117:ld $4,16($fp)dla $2,MM_RTLinkerld $2,0($2) << WRONGld $3,8($fp)move $5,$2move $6,$3jal memcpysd $2,0($fp).stabn 68,0,220,.LM118.LM118:dla $2,MM_RTLinkerld $2,0($2) << WRONGmove $4,$2jal free.stabn 68,0,221,.LM119.LM119:ld $4,16($fp)dla $3,MM_RTLinkerld $2,136($3) << OKandi $2,$2,0x0or $2,$4,$2sd $2,136($3) << OK .stabn 68,0,222,.LM120.LM120:dla $2,MM_RTLinkerld $3,0($2) << WRONGdla $2,MM_RTLinkerld $2,0($2) << WRONGdaddu $4,$3,$2dla $3,MM_RTLinkerld $2,128($3) << OKandi $2,$2,0x0or $2,$2,$4sd $2,128($3) << OK.L65:.LBE9:.stabn 68,0,224,.LM121.LM121:move $sp,$fpld $31,56($sp)ld $fp,48($sp)ld $28,40($sp)daddiu $sp,$sp,64j $31.end RTLinker__ExpandModuleTableHere is corrected code:RTLinker__ExpandModuleTable:.frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0.mask 0xd0000000,-8.fmask 0x00000000,0daddiu $sp,$sp,-64sd $31,56($sp)sd $fp,48($sp)sd $28,40($sp)move $fp,$splui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable)))daddu $28,$28,$25daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))).LBB9:.stabn 68,0,208,.LM109.LM109:sd $0,16($fp).stabn 68,0,210,.LM110.LM110:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2)bne $2,$0,.L61.stabn 68,0,212,.LM111.LM111:dla $2,MM_RTLinkerdaddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4sd $2,0($3).stabn 68,0,213,.LM112.LM112:li $4,4000 # 0xfa0jal mallocsd $2,0($fp)dla $2,MM_RTLinkerdaddiu $3,$2,136 << CORRECT ld $2,0($fp)sd $2,0($3).stabn 68,0,214,.LM113.LM113:dla $2,MM_RTLinkerdaddiu $2,$2,136 << CORRECT ld $2,0($2)bne $2,$0,.L65jal abortb .L65.L61:I still dump core with this change, but I get much further.I have not tested this change at all otherwise -- i.e. on working platforms.I have not debugged through the gcc/m3cg code to see whythe offsets are only sometimes lost, or why other platformshave no problem. Is MIPS's "global pointer" unusual these days?I know SPARC has similar, and all the non-x86, non-AMD64 NT platformshad similar, but that's essentially nothing now (I'm not counting IA64).I know Linux has PLT (procedure linkage table) and GOT (global offset table).Same thing?I haven't looked much at other MIPS ports yet.http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8is what made it look about like it does -- in particularing using BIT_FIELD_REF.1.7:static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T;{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); }#if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); }#else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } }#endif}static voidm3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t;{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); }#if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0);#else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); }#endif EXPR_POP ();}1.8:static voidm3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */{ if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); }}static voidm3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */{ tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP ();}- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 14 10:20:13 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 14 Dec 2008 09:20:13 +0000 Subject: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? In-Reply-To: <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> References: <80AE04BD-4819-4085-8EBE-5996E8E6DB93@cs.purdue.edu> <2014DE82-09E8-46CA-93A6-1A45A0E77268@cs.purdue.edu> Message-ID: Treelang doesn't look hopeful. It is more like a "real language", though incomplete, and less about "direct tree building". I was hoping for something Lisp-like that mapped directly to the internal data structures. For now, I'm tired/lame/impatient, and will just stick with the "* 2" patch. Or maybe when we call relayout_decl, we can have a proper size. Maybe loads of module globals should go through some more specific than m3_load, and we can track the maximum offset. I'll try to write something reasonable to the gcc folks.. I could possibly write my own "treelang", but it doesn't seem worth the time currently, too much else to do. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: FW: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 14 Dec 2008 08:40:05 +0000 (truncated and reedited) Well, they are, definitely, but not with any fields, I think. Like, we say there is a struct with size=align=maxalign, and then we access bit fields an offset from it. If maxalign is large enough, no chance of the struct fitting in a register. I meant, like, a struct where each module global is an actual declared/typed/sized field/member/component/whatever in the struct, not just "randomly" accessing way into and past the struct. I'm delayed now for a day or so but I'll see if I can get a repro using "treelang". Hopefully, going from just its name, it lets you build the exact trees that parse.c builds. As well I should be able to use mips64/netbsd or mips64/linux. OpenBSD isn't supported in current gcc without small patches (which cm3 applies). And then ask the gcc folks (cc you/all?) I can also try reproing with x86/AMD64/ppc lowering maxalign to 32 or 64. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 14 Dec 2008 17:23:06 +1100 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? I thought module globals *were* declared as a struct -- it's been a while... On 14 Dec 2008, at 16:53, Jay wrote: I tried getting C to error this way, and failed. It looks like C uses COMPONENT_REF here, which will send it down different paths I didn't follow. COMPONENT_REF makes sense for it and only somewhat sense for us. That would involve declaring all of a module's globals a struct, which I don't believe we currently do but might be good. I think there's a basic disconnect in that in C, bitfields are always fields, never freestanding. Modula-3 rather violates that. Seems a little dangerous therefore to be using them this way? In terms of the inefficient option in parse.c, that would maybe go away, as all the offseting would be handled by the next level down, like, implementation of fields? Well..that is assuming the only use offset != 0 in m3_load is for accessing module globals. I don't know if that is true. Maybe another option, should be easy to implement but might compile slowly, is to cons up a field decl right there in m3_load/store? That would lead to a lot of extra nodes -- per access instead of declared once. I can try that out. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sat, 13 Dec 2008 18:03:08 +0000 I've just spent a few hours debugging this. The problem is strongly related to the declaration that the global segment is of size BIGGEST_ALIGNMENT. gcc decides they can fit in a register..so then it doesn't bother adding the offset to the address, something like that. When things work, it goes here: op0 = adjust_address (op0, mode1, offset); in expr.c. When things don't work, that line is skipped and then shortly below that: if (mode1 != mode) return convert_to_mode (tmode, op0, unsignedp); return op0; << On systems where BIGGEST_ALIGNMENT fits in a register, e.g. apparently MIPS, counter-e.g. apparently many others, but not clearly all others. This determines whether the mode of the segment is SImode (32bit integer), DImode (64bit integer) or BLKmode (larger than any machine type?). BLKmode works. It works on 32bit MIPS because between mode1 and mode, one of them is SImode, one is DImode. On x86/AMD64/PowerPC systems, I expect it is BLKmode, since biggest alignment is set very large -- 128. I don't know why it works for SPARC. PERHAPS I can put together a C or treelang example and get more (gcc developers) eyes on it, but I doubt it. The following hack works around it: C:\dev2\cm3.2\m3-sys\m3cc\gcc\gcc\m3cg>cvs -z3 diff parse.c Index: parse.c =================================================================== RCS file: /usr/cvs/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c,v retrieving revision 1.77 diff -r1.77 parse.c 2865c2865 < = m3_build_type (T_struct, BIGGEST_ALIGNMENT, BIGGEST_ALIGNMENT); --- > = m3_build_type (T_struct, BIGGEST_ALIGNMENT * 2, BIGGEST_ALIGNMENT); The repro is very easy. Just add this to m3core: MODULE Main2 EXPORTS Main; IMPORT RTIO; VAR a := "a"; BEGIN RTIO.PutText(a); END Main2. The load of a will be from MM_Main2 offset 0, instead of 104 or thereabouts. I'll commit this shortly. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu Date: Thu, 13 Nov 2008 00:04:32 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? > Jay, do you have access to an AMD64_DARWIN box? No, sorry, I don't yet and not sure when I will. Unless it is easy enough to run on non-Apple hardware. I'll have to debug it (m3cg) then. - Jay CC: m3devel at elegosoft.com From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Wed, 12 Nov 2008 20:46:53 +0000 Jay, do you have access to an AMD64_DARWIN box? I did get things working on that 64-bit platform. Or is it an endianness issue you are dealing with. In any case, I will not have a lot of bandwidth for doing much in the way of tackling this unless it becomes critical path for me soon. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 12 Nov 2008, at 07:57, Jay wrote: Target.Little_endian is used to drive some target-specificity in the IR. I guess, after all, that is why you have to specify it. The IR for RTLinker.m3 is identical for MIPS64_OPENBSD and SPARC64_SOLARIS, once I set them both to unaligned_procedures, which they both need. (Tangent, I ran m3cgcat on the .ms file a few times instead of the .mc file; that makes it crash.) I can't claim SPARC64_SOLARIS is any paragon of working and correct though, by far. I should also be able to compare with SPARC64_OPENBSD, I forgot about that, and didn't have an m3cg. I only ran SPARC64_SOLARIS far enough to discover it needed unaligned procedures. I think I should fork linux-common and solaris-common off of the minimal 29k openbsd-common, rather than worry about all the stuff in solaris-2-x and linux-libc, over 100k each. Then SPARC64_SOLARIS could/would be in better shape probably. MIPS64_OPENBSD cm3 gets further with unaligned_procedures, along with the parse.c fix -- it gets to a bus error in RunMainBody. Again an alignment problem, though I think maybe memory memory trashing. The fault occurs here: desc_offset := init_depth * ADRSIZE (InitDesc); desc := init_stack + desc_offset; init_depth := desc.low_link; << InitDesc = RECORD module : RT0.ModulePtr; low_link : INTEGER; END; desc_offset appears to not be a multiple of 4. I will look into that. Long story short -- consider again the parse.c change? I admit I still haven't debugged it (or rather, the code that consumes the data produced by it). - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Date: Sun, 9 Nov 2008 11:10:54 +0000 I agree. I am suspicious too. I will investigate more. Thanks. - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sun, 9 Nov 2008 10:01:58 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Aren't the integer values supposed to be written endian-independently in the IR? I am suspicious about these differences. On 9 Nov 2008, at 01:25, Jay wrote: Thanks for the tip/direction. Uncertain -- m3cgcat between AMD64_LINUX and MIPS64_OPENBSD are "the same" but for two systematic diffs: - AMD64_LINUX has a bunch of added calls to check_nil. This is a "bug" in it, inoptimal but ok code, because "first readable address" is 0. It should probably be 4k. I should write a program that iterates starting at 0 to find the value. (or better yet, cm3 should have a -configure option that does this? -- for configuration values for which there is a safe default; in general, imho, the targets should be auto configured via C, assuming there is a native development environment available). - some integers are different (again, systematically) AMD64_LINUX: init_int 656 13 Int.64 init_chars 664 " typecode " init_int 680 2 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 2 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 2 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 2 Int.64 init_var 816 v.1 0 init_int 824 17 Int.64 init_chars 832 "ExpandInitStack: " MIPS64_OPENBSD: init_chars 664 " typecode " init_int 680 8796093022208 Int.64 init_var 688 v.1 0 init_int 696 13 Int.64 init_chars 704 " typeid " init_int 720 8796093022208 Int.64 init_var 728 v.1 0 init_int 736 13 Int.64 init_chars 744 "RunMainBody: " init_int 760 8796093022208 Int.64 init_var 768 v.1 0 init_int 776 19 Int.64 init_chars 784 "RunMainBody: exec: " init_int 808 8796093022208 Int.64 init_var 816 v.1 0 This 8796093022208 is 80000000000 aka 0000_0800_0000_0000, where little endian has 2. I don't know what these numbers represent. I should look at other systems -- preferably a working 64 bit big endian system, if there are any. Could be flags/type with text literals? (Btw, I always leave the TextLiteral.i3 change on my system, lowering the maximum literal length to around 4gig even on 64 bit targets, so 32 bit can bootstrap 64 bit...need to fix the compiler here to do target arithmetic instead of host arithmetic...) Thanks, - Jay From: hosking at cs.purdue.edu To: jay.krell at cornell.edu Date: Sat, 8 Nov 2008 22:55:54 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] m3_load/m3_store/bit_field_ref lose offset?? Have you checked to make sure that the pre-gcc IR makes sense? Does it see these values as 32-bit or 64-bit? The reason I ask is that there may be something wrong in your m3middle target specs that cause bad IR. Once we know the IR looks right then we can try and fix the problem. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On 8 Nov 2008, at 15:45, Jay wrote: truncated as usual...along with losing a lot of whitespace.. Let's see... Tony, how about switching these #if's? I explain some of why below. static void m3_store (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { tree val; #if 0 if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != dst_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, dst_t, m3_cast (build_pointer_type (dst_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ val = m3_cast (src_t, EXPR_REF (-1)); if (src_T != dst_T) { val = m3_build1 (CONVERT_EXPR, dst_t, val); } add_stmt (build2 (MODIFY_EXPR, dst_t, v, val)); EXPR_POP (); } m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) { #if 0 if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o)); } #else /* failsafe, but inefficient */ if (o != 0 || TREE_TYPE (v) != src_t) { v = m3_build1 (ADDR_EXPR, t_addr, v); v = m3_build2 (PLUS_EXPR, t_addr, v, size_int (o / BITS_PER_UNIT)); v = m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), v)); } #endif TREE_THIS_VOLATILE (v) = 1; /* force this to avoid aliasing problems */ if (src_T != dst_T) { v = m3_build1 (CONVERT_EXPR, dst_t, v); } EXPR_PUSH (v); } I'm working on MIPS64_OPENBSD (OpenBSD on an SGI O2). In an early version it seems that many loads/stores of global variables is wrong. The "first" code affected is RTLinker_ExpandModuleTable. PROCEDURE ExpandModuleTable () = CONST InitialTableSize = 500; VAR new_mods: ADDRESS; n_bytes: INTEGER; BEGIN IF (modules = NIL) THEN (* first time... *) max_modules := InitialTableSize; modules := Cstdlib.malloc (InitialTableSize * BYTESIZE (RT0.ModulePtr)); IF (modules = NIL) THEN Cstdlib.abort (); END; ELSE n_bytes := max_modules * BYTESIZE (RT0.ModulePtr); new_mods := Cstdlib.malloc (n_bytes + n_bytes); IF (new_mods = NIL) THEN Cstdlib.abort (); END; EVAL Cstring.memcpy (new_mods, modules, n_bytes); Cstdlib.free (modules); modules := new_mods; INC (max_modules, max_modules); END; END ExpandModuleTable; It should read back modules as NIL. What happens is it reads back modules as not NIL, and then max_modules should also be zero, but is "large", leading to malloc to fail, and abort called. The output of cm3cg -y is reasonable. It indicates offset loads. What is incorrect is that in many loads and stores, but not all, the offsets are lost. (923) begin_procedure procedure RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable(924) set_source_line source line 208 (925) load_nil (926) store store (M3_AJWxb1_new_mods) offset 0x0 src_t 0xb dst_t 0xb (927) set_source_line source line 207 (928) set_source_line source line 210 (929) load_nil (930) load m3cg_load (MM_RTLinker): offset 0x440, convert 0xb -> 0xb Here is the incorrect code: .globl RTLinker__ExpandModuleTable .stabd 46,0,0 .stabn 68,0,206,.LM108 .LM108: .LFBB8: .set nomips16 .ent RTLinker__ExpandModuleTable RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $3,MM_RTLinker ld $2,128($3) andi $2,$2,0x0 ori $2,$2,0x1f4 sd $2,128($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) ld $4,0($fp) dla $3,MM_RTLinker ld $2,136($3) andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK, offset seems low, but it agrees with elsewhere .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker ld $2,0($2) << WRONG bne $2,$0,.L65 jal abort b .L65 .L61: .stabn 68,0,216,.LM114 .LM114: dla $2,MM_RTLinker ld $2,0($2) << WRONG dsll $2,$2,3 sd $2,8($fp) .stabn 68,0,217,.LM115 .LM115: ld $3,8($fp) ld $2,8($fp) daddu $2,$3,$2 move $4,$2 jal malloc sd $2,0($fp) ld $2,0($fp) sd $2,16($fp) .stabn 68,0,218,.LM116 .LM116: ld $2,16($fp) bne $2,$0,.L64 jal abort .L64: .stabn 68,0,219,.LM117 .LM117: ld $4,16($fp) dla $2,MM_RTLinker ld $2,0($2) << WRONG ld $3,8($fp) move $5,$2 move $6,$3 jal memcpy sd $2,0($fp) .stabn 68,0,220,.LM118 .LM118: dla $2,MM_RTLinker ld $2,0($2) << WRONG move $4,$2 jal free .stabn 68,0,221,.LM119 .LM119: ld $4,16($fp) dla $3,MM_RTLinker ld $2,136($3) << OK andi $2,$2,0x0 or $2,$4,$2 sd $2,136($3) << OK .stabn 68,0,222,.LM120 .LM120: dla $2,MM_RTLinker ld $3,0($2) << WRONG dla $2,MM_RTLinker ld $2,0($2) << WRONG daddu $4,$3,$2 dla $3,MM_RTLinker ld $2,128($3) << OK andi $2,$2,0x0 or $2,$2,$4 sd $2,128($3) << OK .L65: .LBE9: .stabn 68,0,224,.LM121 .LM121: move $sp,$fp ld $31,56($sp) ld $fp,48($sp) ld $28,40($sp) daddiu $sp,$sp,64 j $31 .end RTLinker__ExpandModuleTable Here is corrected code: RTLinker__ExpandModuleTable: .frame $fp,64,$31 # vars= 32, regs= 3/0, args= 0, gp= 0 .mask 0xd0000000,-8 .fmask 0x00000000,0 daddiu $sp,$sp,-64 sd $31,56($sp) sd $fp,48($sp) sd $28,40($sp) move $fp,$sp lui $28,%hi(%neg(%gp_rel(RTLinker__ExpandModuleTable))) daddu $28,$28,$25 daddiu $28,$28,%lo(%neg(%gp_rel(RTLinker__ExpandModuleTable))) .LBB9: .stabn 68,0,208,.LM109 .LM109: sd $0,16($fp) .stabn 68,0,210,.LM110 .LM110: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT (offset seems low, but at least it isn't zero) ld $2,0($2) bne $2,$0,.L61 .stabn 68,0,212,.LM111 .LM111: dla $2,MM_RTLinker daddiu $3,$2,128 << CORRECT li $2,500 # 0x1f4 sd $2,0($3) .stabn 68,0,213,.LM112 .LM112: li $4,4000 # 0xfa0 jal malloc sd $2,0($fp) dla $2,MM_RTLinker daddiu $3,$2,136 << CORRECT ld $2,0($fp) sd $2,0($3) .stabn 68,0,214,.LM113 .LM113: dla $2,MM_RTLinker daddiu $2,$2,136 << CORRECT ld $2,0($2) bne $2,$0,.L65 jal abort b .L65 .L61: I still dump core with this change, but I get much further. I have not tested this change at all otherwise -- i.e. on working platforms. I have not debugged through the gcc/m3cg code to see why the offsets are only sometimes lost, or why other platforms have no problem. Is MIPS's "global pointer" unusual these days? I know SPARC has similar, and all the non-x86, non-AMD64 NT platforms had similar, but that's essentially nothing now (I'm not counting IA64). I know Linux has PLT (procedure linkage table) and GOT (global offset table). Same thing? I haven't looked much at other MIPS ports yet. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.7;r2=1.8 is what made it look about like it does -- in particularing using BIT_FIELD_REF. 1.7: static void m3_load (v, o, src_t, src_T, dest_t, dest_T) /* GCC32OK */ tree v; int o; tree src_t, dest_t; m3_type src_T, dest_T; { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { tree adr = m3_build1 (ADDR_EXPR, t_addr, v); if (o != 0) { adr = m3_build2 (PLUS_EXPR, t_addr, adr, size_int (o / BITS_PER_UNIT)); } EXPR_PUSH (m3_build1 (INDIRECT_REF, src_t, m3_cast (build_pointer_type (src_t), adr))); } #if 1 if (src_T != dest_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dest_t, EXPR_REF (-1)); } if (debug_vars) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dest_T); } #else if (src_T != dest_T) { if (IS_INTEGER_TYPE(dest_T) && dest_t != t_int) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_int, EXPR_REF (-1)); } else if (IS_WORD_TYPE(dest_T) && dest_t != t_word) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, t_word, EXPR_REF (-1)); } else if (IS_INTEGER_TYPE(dest_T) || IS_WORD_TYPE(dest_T)) { /* okay */ } else { fatal_error("m3load: cannot convert types: src_t %d dest_t %d\n", src_T, dest_T); } } #endif } static void m3_store (v, o, src_t, dest_t) /* GCC32OK */ tree v; int o; tree src_t, dest_t; { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dest_t) { lhs = v; } else { tree f = make_node (FIELD_DECL); TREE_TYPE (f) = dest_t; DECL_ALIGN (f) = TYPE_ALIGN (dest_t); DECL_SIZE (f) = TYPE_SIZE (dest_t); DECL_MODE (f) = TYPE_MODE (dest_t); DECL_FIELD_OFFSET (f) = size_int (o / BITS_PER_UNIT); DECL_FIELD_BIT_OFFSET (f) = bitsize_int (o % BITS_PER_UNIT); DECL_FIELD_CONTEXT (f) = TREE_TYPE (v); lhs = m3_build2 (COMPONENT_REF, dest_t, v, f); } #if 1 expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); #else if (dest_t == NULL) { expand_assignment (lhs, rhs, 0, 0); } else { expand_assignment (lhs, m3_build1 (CONVERT_EXPR, dest_t, rhs), 0, 0); } #endif EXPR_POP (); } 1.8: static void m3_load (tree v, int o, tree src_t, m3_type src_T, tree dst_t, m3_type dst_T) /* GCC32OK */ { if (o == 0 && TREE_TYPE (v) == src_t) { EXPR_PUSH (v); } else { EXPR_PUSH (m3_build3 (BIT_FIELD_REF, src_t, v, TYPE_SIZE (src_t), bitsize_int (o))); } if (src_T != dst_T) { EXPR_REF (-1) = m3_build1 (CONVERT_EXPR, dst_t, EXPR_REF (-1)); } if (option_vars_trace) { const char *name = "noname"; if (v != 0 && DECL_NAME(v) != 0) { name = IDENTIFIER_POINTER(DECL_NAME(v)); } fprintf(stderr, " m3_load (%s): offset %d, convert %d -> %d\n", name, o, src_T, dst_T); } } static void m3_store (tree v, int o, tree src_t, tree dst_t) /* GCC32OK */ { tree lhs, rhs; if (TREE_TYPE (EXPR_REF (-1)) == src_t) { rhs = EXPR_REF (-1); } else { rhs = m3_cast (src_t, EXPR_REF (-1)); } if (o == 0 && TREE_TYPE (v) == dst_t) { lhs = v; } else { lhs = m3_build3 (BIT_FIELD_REF, dst_t, v, TYPE_SIZE (dst_t), bitsize_int (o)); } if (src_t != dst_t) { rhs = m3_build1 (CONVERT_EXPR, dst_t, rhs); } expand_assignment (lhs, rhs, 0, 0); EXPR_POP (); } - Jay From jay.krell at cornell.edu Mon Dec 15 17:37:33 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 15 Dec 2008 16:37:33 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know. Not quite eveything is checked in and I didn't build a distribution yet. Within a few days, hopefully. I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 12:32:18 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 11:32:18 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass. Two are overly 32 bit specific. Two hang. Others fail that look familiar and/or floating point related, so probably fail on all. I didn't look too closely. The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap? I can try going back to that before debugging the hangs. In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited. Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 16 17:22:52 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 16 Dec 2008 16:22:52 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I fixed the hang and updated the two archives. I had time_t wrong in some places. Are people on this list using any of: Linux/AMD64 FreeBSD/AMD64 NetBSD/AMD64 OpenBSD/AMD64 Solaris/AMD64 or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? or for that matter...? (I use Windows/AMD64 but am holding off on anything there.) - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Tue, 16 Dec 2008 11:32:18 +0000 please try out: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz and/or http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz Regression tests mostly pass.Two are overly 32 bit specific.Two hang.Others fail that look familiar and/or floating point related, so probably fail on all.I didn't look too closely.The hangs need investigation. Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs.In one case I know the test didn't hang, but the test infra structure hangs waiting for it. All relevant changes are commited.Sorry this has taken so long.. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comSubject: RE: [M3devel] AMD64_LINUX (again)Date: Mon, 15 Dec 2008 16:37:33 +0000 I never debugged the formsedit crash, but it seems to be gone now. Possibly courtesy of pruning down the cloned headers, I don't know.Not quite eveything is checked in and I didn't build a distribution yet.Within a few days, hopefully.I need to try the regression tests too. - Jay From: jay.krell at cornell.eduTo: m3devel at elegosoft.comDate: Mon, 17 Nov 2008 08:39:48 +0000Subject: [M3devel] AMD64_LINUX (again) ok, with the sbrk workaround now: http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz You don't have to build anything. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz You probably want to run ./do-cm3-std.py buildship. http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz You get to build your own of almost everything (./install-config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py buildship && ./install-front.py && ./do-cm3-std.py buildship) OR you can bootstrap from an existing other system, roughly: cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target machine and extract it, cd into it, make, and then you have native cm3 and proceed to previous.. I'm able to run Juno, Cube, Calculator, Mentor. Hm. formsedit crashes. Darn. Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 1140881760 (LWP 13270)]0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325325 VBT.PaintTint(v, r, v.shadow.bg);(gdb) cContinuing.Cannot find user-level thread for LWP 13242: generic error(gdb) bt#0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows (M3_Ao3CED_v=Cannot access memory at address 0x144006bf8) at ../src/lego/POSIX/ScrollerVBTClass.m3:325Cannot fetch general-purpose registers for thread 1140881760: generic error(gdb) still might be worth other people trying out.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Tue Dec 16 20:33:27 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:33:27 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081215173318.0398710D5C74@birch.elegosoft.com> References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: This new file FilePosixC.c does not build on my SOLgnu Tinderbox setup, so the regressions failed last night. Jay, this is precisely the reason that we don't want to have C-coded files in the Modula-3 code-base -- C might proclaim its portability but that portability is a lie. Why do we need this when the old pure Modula-3 setup worked just fine! I know it meant some amount of duplication, but that duplication was needed because of differences in API specs on different platforms. I am not averse to having *portable* C code, but in this case I think it is too low-level to be properly portable. Perhaps better to keep it in Modula-3. On 15 Dec 2008, at 18:33, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/15 18:33:17 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > Log message: > Remove the need to untangle the #ifdefs around struct flock. From hosking at cs.purdue.edu Tue Dec 16 20:39:17 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 06:39:17 +1100 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: On 16 Dec 2008, at 22:32, Jay wrote: > please try out: > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > and/or > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > > Regression tests mostly pass. > Two are overly 32 bit specific. > Two hang. > Others fail that look familiar and/or floating point related, so > probably fail on all. > I didn't look too closely. > The hangs need investigation. > > Tony, should mmap work now, for arbitrarily distributed mmap? > I can try going back to that before debugging the hangs. Probably not. We still need a byte-sized entry per heap page that is maintained in the old array. I am working on developing a sparser representation for the 2 bits of information we still need to be able to extract given an arbitrary ambiguous root (i.e., something that looks like a pointer into the heap from the thread stacks). > > In one case I know the test didn't hang, but the test infra > structure hangs waiting for it. > > All relevant changes are commited. > Sorry this has taken so long.. > > - Jay > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Subject: RE: [M3devel] AMD64_LINUX (again) > Date: Mon, 15 Dec 2008 16:37:33 +0000 > > I never debugged the formsedit crash, but it seems to be gone now. > Possibly courtesy of pruning down the cloned headers, I don't know. > Not quite eveything is checked in and I didn't build a distribution > yet. > Within a few days, hopefully. > I need to try the regression tests too. > > - Jay > > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Mon, 17 Nov 2008 08:39:48 +0000 > Subject: [M3devel] AMD64_LINUX (again) > > ok, with the sbrk workaround now: > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-std-AMD64_LINUX-d5.7.0.tar.gz > You don't have to build anything. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-min-AMD64_LINUX-d5.7.0.tar.gz > You probably want to run ./do-cm3-std.py buildship. > > > http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-AMD64_LINUX.gz > You get to build your own of almost everything (./install- > config.py && ./do-pkg.py m3cc buildship && ./do-cm3-front.py > buildship && ./install-front.py && ./do-cm3-std.py buildship) > > > OR you can bootstrap from an existing other system, roughly: > cd m3-sys/m3cc && cm3 -DM3CC_TARGET=AMD64_LINUX && cd scripts/ > python && ./do-cm3-front.py AMD64_LINUX boot && ./boot1.py > AMD64_LINUX and copy /cm3-boot-AMD64_LINUX-1.tar.gz to target > machine and extract it, cd into it, make, and then you have native > cm3 and proceed to previous.. > > > I'm able to run Juno, Cube, Calculator, Mentor. > > Hm. formsedit crashes. Darn. > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 1140881760 (LWP 13270)] > 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Cannot > access memory at address 0x144006bf8 > ) > at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > 325 VBT.PaintTint(v, r, v.shadow.bg); > (gdb) c > Continuing. > Cannot find user-level thread for LWP 13242: generic error > (gdb) bt > #0 0x00002afc180b9d6a in ScrollerVBTClass__PaintViewWithShadows > (M3_Ao3CED_v=Ca > nnot access memory at address 0x144006bf8 > ) at ../src/lego/POSIX/ScrollerVBTClass.m3:325 > Cannot fetch general-purpose registers for thread 1140881760: > generic error > (gdb) > > > still might be worth other people trying out.. > > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 01:32:06 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 00:32:06 +0000 Subject: [M3devel] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: Tony, Well, it is a matter of degree, but I really dislike the duplication and I believe the portability is plenty attainable. It also didn't compile for me on Cygwin, I did a bit more research and fixed that. It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't rebuild there yet. L_SET is not portable. - Jay> From: hosking at cs.purdue.edu> To: jkrell at elego.de> Date: Wed, 17 Dec 2008 06:33:27 +1100> CC: m3devel at elegosoft.com; m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3> > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > setup, so the regressions failed last night.> > Jay, this is precisely the reason that we don't want to have C-coded > files in the Modula-3 code-base -- C might proclaim its portability > but that portability is a lie. Why do we need this when the old pure > Modula-3 setup worked just fine! I know it meant some amount of > duplication, but that duplication was needed because of differences in > API specs on different platforms. I am not averse to having > *portable* C code, but in this case I think it is too low-level to be > properly portable. Perhaps better to keep it in Modula-3.> > On 15 Dec 2008, at 18:33, Jay Krell wrote:> > > CVSROOT: /usr/cvs> > Changes by: jkrell at birch. 08/12/15 18:33:17> >> > Modified files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3> > m3makefile> > Added files:> > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c> >> > Log message:> > Remove the need to untangle the #ifdefs around struct flock.> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 01:49:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 17 Dec 2008 11:49:36 +1100 Subject: [M3devel] [M3commit] FilePosixC.c In-Reply-To: References: <20081215173318.0398710D5C74@birch.elegosoft.com> Message-ID: I'll wait for tomorrow's regressions to come through and see if it is fixed now for my SOLgnu Tinderbox setup. I see the temptation to eliminate duplication, and for these sorts of POSIX APIs I would expect portable C code to work pretty well. On 17 Dec 2008, at 11:32, Jay wrote: > Tony, Well, it is a matter of degree, but I really dislike the > duplication > and I believe the portability is plenty attainable. It also didn't > compile > for me on Cygwin, I did a bit more research and fixed that. > It is ok now for SOLgnu? I do have a SOLgnu/sun machine but didn't > rebuild there yet. > L_SET is not portable. > > - Jay > > > From: hosking at cs.purdue.edu > > To: jkrell at elego.de > > Date: Wed, 17 Dec 2008 06:33:27 +1100 > > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > This new file FilePosixC.c does not build on my SOLgnu Tinderbox > > setup, so the regressions failed last night. > > > > Jay, this is precisely the reason that we don't want to have C-coded > > files in the Modula-3 code-base -- C might proclaim its portability > > but that portability is a lie. Why do we need this when the old pure > > Modula-3 setup worked just fine! I know it meant some amount of > > duplication, but that duplication was needed because of > differences in > > API specs on different platforms. I am not averse to having > > *portable* C code, but in this case I think it is too low-level to > be > > properly portable. Perhaps better to keep it in Modula-3. > > > > On 15 Dec 2008, at 18:33, Jay Krell wrote: > > > > > CVSROOT: /usr/cvs > > > Changes by: jkrell at birch. 08/12/15 18:33:17 > > > > > > Modified files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > > > m3makefile > > > Added files: > > > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.c > > > > > > Log message: > > > Remove the need to untangle the #ifdefs around struct flock. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 02:00:35 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 01:00:35 +0000 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: I asked mainly in case it was related to the hang, but it wasn't. Thanks, - Jay CC: m3devel at elegosoft.comFrom: hosking@ Tony, should mmap work now, for arbitrarily distributed mmap?I can try going back to that before debugging the hangs. Probably not. ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 16 18:26:05 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 16 Dec 2008 11:26:05 -0600 Subject: [M3devel] AMD64_LINUX (again) In-Reply-To: References: Message-ID: <4947E4AD.8080705@wichita.edu> I am using Linux/AMD64. Jay wrote: > I fixed the hang and updated the two archives. > I had time_t wrong in some places. > > Are people on this list using any of: > Linux/AMD64 > FreeBSD/AMD64 > NetBSD/AMD64 > OpenBSD/AMD64 > Solaris/AMD64 > or for that matter, {NetBSD,OpenBSD}/*, Solaris/x86? > or for that matter...? > > (I use Windows/AMD64 but am holding off on anything there.) > > - Jay > From rcoleburn at scires.com Wed Dec 17 16:12:09 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 10:12:09 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> Message-ID: <4948D035.1E75.00D7.1@scires.com> Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 17:29:54 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 16:29:54 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4948D035.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:10:05 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:10:05 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 21:23:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 07:23:06 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> Message-ID: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: > Eh, not a big deal, you can cd to it and it just fine. > > I removed it from my distribution building and regular building. > It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering > wasn't working. > I "merely" need to fix one or the other or just comment it out when > building Cygwin, which is rare. > > See...I figured..that the Cygwin serial package should still use the > Win32 source. > However, that has a dependency on FileWin32.m3, which is only built > in "full Win32" platforms. > If you do try to compile FileWin32.m3, it errors, because it exports > File, and I guess maybe FileWin32. > Possibly I just need to have Quake do a bit of munging such that the > first line of FileWin32.m3 is altered. > The NT386 version would be: > MODULE FileWin32 EXPORTS File. > > The NT386GNU version would be: > MODULE FileWin32; > > something like that. > > Or push the contents into another module, and then have a thin NT386 > module export it as File. > > Anyway, it's not a huge deal. > The main downfall is I "temporarily" no longer build it, on any > platform. > > Fixing the filtering is a better "temporary" solution, since that'd > restore me to building it for all but NT386GNU. > > Another avenue is to see if the Posix version works on Cygwin. > I think the Cygwin serial users however are a "multiple small > intersection" -- people who use Modula-3 (small) intersected with > people who use Cygwin (medium) (probably at zero here) intersected > with people who use serial (very small) => very very very small, so > that's not worth getting it to work. > > - Jay > > > Date: Wed, 17 Dec 2008 10:12:09 -0500 > From: rcoleburn at scires.com > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > > Jay: > > Why are we skipping serial package? I use serial I/O on Windows. > If you need my sources, let me know and I'll supply. > > Regards, > Randy > > >>> Jay Krell 12/17/2008 3:47 PM >>> > CVSROOT:/usr/cvs > Changes by:jkrell at birch.08/12/17 15:47:34 > > Modified files: > cm3/scripts/python/: pylib.py make-dist.py > > Log message: > fix line endings and 'temporarily' skip serial package -- there's a > problem here on I386_CYGWIN I forgot about where essentially you > want to provide both FilePosix and FileWin32; FilePosix will export > File, FileWin32 will export FileWin32, so that portable Modula-3 can > use File/FilePosix and Win32 specific serial code can use FileWin32; > maybe some day we should try out this serial stuff, and /maybe/ the > Posix paths work on Cygwin? For now, just skip it. And fix line > endings. And try out the alternate 'clean' path in make-dist > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 22:36:36 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 21:36:36 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> Message-ID: I have a SOLsun/SOLgnu system at home. Wait a few more hours please? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:05:30 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:05:30 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: From: TonyTo: jay I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? It's more complicated than that. It isn't just one bit. There's a few factors, that people care more and less about. GUI library -- Win32 or X thread library -- pthreads or Win32 (or user threads) file system path representation -- forward slashes or backward slashes This seems to be what people care most about -- to see forward slashes. C compiler -- Visual C++ cl or Cygwin/GNU gcc linker -- Visual C++ link or Cygwin/GNU ld C runtime -- msvcr*.dll or cygwin1.dll integrated backend or not The integrated backend is way faster, but doesn't yet support 64bit LONGINT. which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/pdb Many of these factors are independent of each other, but not entirely. Compiler/linker directly lead to which debugger you can use, since they each use different formats. Well, you can use either debugger, but..without symbols..at least for some modules. The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBase, so you can't link to the Cygwin C runtime with theVisual C++ linker. This is probably fixable at least in one combination withan alias (ability to link Cygwin C runtime with Visual C++ linker; I don'tknow if Cygwin/GNU ld understands aliases). I have also found that Cygwin produces incorrectimport .libs, perhaps their linker somehow accepts them. Cygwin can program the Win32 GUI, but there is an issue with passing 8 byte structs by value to __stdcall functions, the function name isn't mangled correctly and linking fails. It is a bit of a combinatorial explosion.As the system stands, you can control each bit in the config file, andmultiple combinations might work, though I don't run any combinatorial tests. Three combinations are "identified" (named, spoken of) and have had some development testing.NT386, NT386MINGNU (which I think should be called I386_MINGWIN), and NT386GNU (which I think should be called I386_CYGWIN). Furthermore, you can imagine I386_INTERIX and I386_UWIN (I forget if that's the name).Interix and UWIN being two other very viable Posix layers on Windows.Interix has been renamed a few times. It is called "SFU" -- services for Unix, and I don't know,like SUA -- services for unix applications, and I think there's another name.It is a free download from Microsoft for x86 Windows < Vista, and is an optional component in Vista.It has its pluses and minuses..not now. Anyway, in conversation here, one of the main things that people DO NOT care aboutis which thread library is used. I believe it was asserted that PM3/NT386GNU did work, but that it might have used Win32 threads. And maybe never had any GUI? I tried getting pthreads to work, but gave up debugging it.Perhaps a fresh go at debugging it would pan out..but I'm really not too interested.. I also have no interest in user threads. I believe they are never warranted, excepton old systems that don't have kernel threads, like the hypothetical DJGPP port. I never brought up the issue of serial ports.I ASSUMED that almost nobody uses them, and anyone that does use them, doesn'tcare if they are implemented using Cygwin or Win32, as long as they work. I ASSUME there is no interop issue, like getting an int fd from the serial portand passing it to select() or close, or getting a Win32 HANDLE and passing itto WaitForSingleObject, but I could be wrong here, certainly. I didn't look into if Cygwin is complete enough to implement them.I consider serial ports to be kind of out on the fringe..that it wouldn't be surprising if Cygwin didn't implement them..but I didn't look. Heck, I tried building minicom on some non-Linux system(s) recently and it failed miserably. I don't know if there is even much Posix portability option for serial port programming. Sure..I debug over a serial port, and lately have machines I can't get videoout of (Itanium) or even seem to have no video hardware (Sun), so I use a serial consolefor initial install (recent achievement :) still haven't installed over network..), but that's still imho a big step away from writing code that uses the serial port.(And NetBSD doesn't support the local video on SGI O2, but I'm running OpenBSD there for now.) Anyway, there are a few ways to look at this issue.More and less trivial. A trivial view is: Get the filtering to work. Build serial on all platforms but Cygwin. Or see if the Posix code will work. I'd be surprised if the intersection of Cygwin x Modula-3 x serial port programmers is not 0, but could be. I actually don't think anyone is using Cygwin x Modula-3. Another view is: Get the Win32 serial code to work on Cygwin. The code is already there, and buildable, but building in a system that also builds FilePosix.m3 breaks Modula-3 rules, you end up with two implementations of the File interface. What you want, I think, is FilePosix.m3 to provide the File interface, and FileWin32.m3 to provide just its own specific FileWin32 interface. I think this is a simple transform of the code. Anyway, not a big deal...my tone here...is rushed, not rude, sorry.Maybe I'll get the truncation I deserve. :) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:07:33 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:07:33 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> Message-ID: <4949318C.1E75.00D7.1@scires.com> Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy >>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building. It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working. I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source. However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms. If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32. Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered. The NT386 version would be: MODULE FileWin32 EXPORTS File. The NT386GNU version would be: MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal. The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin. I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500 From: rcoleburn at scires.com To: jkrell at elego.de CC: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy >>> Jay Krell 12/17/2008 3:47 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/17 15:47:34 Modified files: cm3/scripts/python/: pylib.py make-dist.py Log message: fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:24:38 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:24:38 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: <7D219C8E-022A-4196-8786-CA8F4278C010@cs.purdue.edu> Yeah, I would have thought of M3 on Cygwin as presenting a POSIX API (even if some system services like threads are implemented under the covers using Cygwin). To that extent, everything other than m3core, where threads live, should compile as a POSIX library. On 18 Dec 2008, at 09:07, Randy Coleburn wrote: > Jay, > > I agree with Tony. > > From my point of view, you are "breaking" the normal Windows install > in order to rectify a problem for Cygwin. I put "breaking" in > quotes because the install will appear to work, but the serial > package will be silently omitted. > > It may not be a "big deal" to you, but it presents another hurdle > for those who want to install on Windows, and this hurdle is > probably not documented yet. > > Regards, > Randy > > >>> Tony Hosking 12/17/2008 3:23 PM >>> > I would have thought of CygWin as a POSIX platform that just happens > to have Windows underneath. Did we go round this roundabout before? > > On 18 Dec 2008, at 03:29, Jay wrote: > >> Eh, not a big deal, you can cd to it and it just fine. >> >> I removed it from my distribution building and regular building. >> It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering >> wasn't working. >> I "merely" need to fix one or the other or just comment it out when >> building Cygwin, which is rare. >> >> See...I figured..that the Cygwin serial package should still use >> the Win32 source. >> However, that has a dependency on FileWin32.m3, which is only built >> in "full Win32" platforms. >> If you do try to compile FileWin32.m3, it errors, because it >> exports File, and I guess maybe FileWin32. >> Possibly I just need to have Quake do a bit of munging such that >> the first line of FileWin32.m3 is altered. >> The NT386 version would be: >> MODULE FileWin32 EXPORTS File. >> >> The NT386GNU version would be: >> MODULE FileWin32; >> >> something like that. >> >> Or push the contents into another module, and then have a thin >> NT386 module export it as File. >> >> Anyway, it's not a huge deal. >> The main downfall is I "temporarily" no longer build it, on any >> platform. >> >> Fixing the filtering is a better "temporary" solution, since that'd >> restore me to building it for all but NT386GNU. >> >> Another avenue is to see if the Posix version works on Cygwin. >> I think the Cygwin serial users however are a "multiple small >> intersection" -- people who use Modula-3 (small) intersected with >> people who use Cygwin (medium) (probably at zero here) intersected >> with people who use serial (very small) => very very very small, so >> that's not worth getting it to work. >> >> - Jay >> >> >> Date: Wed, 17 Dec 2008 10:12:09 -0500 >> From: rcoleburn at scires.com >> To: jkrell at elego.de >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] [M3commit] CVS Update: cm3 >> >> >> Jay: >> >> Why are we skipping serial package? I use serial I/O on Windows. >> If you need my sources, let me know and I'll supply. >> >> Regards, >> Randy >> >> >>> Jay Krell 12/17/2008 3:47 PM >>> >> CVSROOT:/usr/cvs >> Changes by:jkrell at birch.08/12/17 15:47:34 >> >> Modified files: >> cm3/scripts/python/: pylib.py make-dist.py >> >> Log message: >> fix line endings and 'temporarily' skip serial package -- there's a >> problem here on I386_CYGWIN I forgot about where essentially you >> want to provide both FilePosix and FileWin32; FilePosix will export >> File, FileWin32 will export FileWin32, so that portable Modula-3 >> can use File/FilePosix and Win32 specific serial code can use >> FileWin32; maybe some day we should try out this serial stuff, and / >> maybe/ the Posix paths work on Cygwin? For now, just skip it. And >> fix line endings. And try out the alternate 'clean' path in make-dist >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Dec 17 23:27:36 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 09:27:36 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> Message-ID: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e- mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 17 23:30:07 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 17 Dec 2008 17:30:07 -0500 Subject: [M3devel] serial2 package Message-ID: <494936D7.1E75.00D7.1@scires.com> Jay: I'm too busy right now to jump into the serial I/O stuff, so I am sending you a ZIP file containing my "serial2" package. This package is a replacement for the regular "serial" package. It has a WIN32 and a POSIX implementation, but the POSIX implementation was tailored for HP-UX and to my knowledge never tested on anything else. The WIN32 implementation works for me. It has some bug fixes and extras that the regular serial package does not and it has been tested on NT, 2000, and XP. You are welcome to use any part of this that is helpful. I know that emailing ZIP files is probably bad form these days, but you can scan it with your AV program. There are no binaries in here, just text files. Regards, Randy Coleburn -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: serial2.zip Type: application/x-zip-compressed Size: 24875 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Dec 17 23:32:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:32:53 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <4949318C.1E75.00D7.1@scires.com> References: <20081217144734.7E9A310D5C1D@birch.elegosoft.com> <4948D035.1E75.00D7.1@scires.com> <42B94D53-2A06-45B5-915C-AFB87D5F7784@cs.purdue.edu> <4949318C.1E75.00D7.1@scires.com> Message-ID: I really don't think it's a big deal, to omit a minor package here or there. Imho people should be able to build whatever they need from source. Granted, I'm not where I want to be there yet, building the OS and such. I think most of the packages have no users at all. The web browsers in the tree for example seemed useless. I don't think Cygwin has any users either, so granted, this compromise was backwards. I could just comment it out for building Cygwin and not commit it. Anyway, I'll improve serial one way or the other. Like either just put it back, breaking Cygwin (if you use the .py files), or fix the filtering (could always filter in the m3makefile, easy). Randy, Tony's point I took to be different. His point is just let it build the Posix code. And indeed, I don't remember if I ever tried it. I just assumed serial programming to be a bit out on the fringe and not surprising if Cygwin doesn't support it, or even if it works on many Posix platforms at all. Serial is after all one of the packages with filtering "built in" just for it. I know minicom (just an example of a friendly serial app on GNU/Linux (e.g. Debian)) for example doesn't compile on some "Posix" platforms I have -- I forgot which I tried..could be any of AIX, Irix, OpenBSD... (was trying to bring up additional serial consoles besides a really noisy machine; the clear winner was Hyperterm on XP using a $20 USB to serial converter fyi...). - Jay Date: Wed, 17 Dec 2008 17:07:33 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay, I agree with Tony. >From my point of view, you are "breaking" the normal Windows install in order to rectify a problem for Cygwin. I put "breaking" in quotes because the install will appear to work, but the serial package will be silently omitted. It may not be a "big deal" to you, but it presents another hurdle for those who want to install on Windows, and this hurdle is probably not documented yet. Regards, Randy>>> Tony Hosking 12/17/2008 3:23 PM >>> I would have thought of CygWin as a POSIX platform that just happens to have Windows underneath. Did we go round this roundabout before? On 18 Dec 2008, at 03:29, Jay wrote: Eh, not a big deal, you can cd to it and it just fine. I removed it from my distribution building and regular building.It fails to build on I386_CYGWIN (aka NT386GNU), AND my filtering wasn't working.I "merely" need to fix one or the other or just comment it out when building Cygwin, which is rare. See...I figured..that the Cygwin serial package should still use the Win32 source.However, that has a dependency on FileWin32.m3, which is only built in "full Win32" platforms.If you do try to compile FileWin32.m3, it errors, because it exports File, and I guess maybe FileWin32.Possibly I just need to have Quake do a bit of munging such that the first line of FileWin32.m3 is altered.The NT386 version would be:MODULE FileWin32 EXPORTS File. The NT386GNU version would be:MODULE FileWin32; something like that. Or push the contents into another module, and then have a thin NT386 module export it as File. Anyway, it's not a huge deal.The main downfall is I "temporarily" no longer build it, on any platform. Fixing the filtering is a better "temporary" solution, since that'd restore me to building it for all but NT386GNU. Another avenue is to see if the Posix version works on Cygwin.I think the Cygwin serial users however are a "multiple small intersection" -- people who use Modula-3 (small) intersected with people who use Cygwin (medium) (probably at zero here) intersected with people who use serial (very small) => very very very small, so that's not worth getting it to work. - Jay Date: Wed, 17 Dec 2008 10:12:09 -0500From: rcoleburn at scires.comTo: jkrell at elego.deCC: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Why are we skipping serial package? I use serial I/O on Windows. If you need my sources, let me know and I'll supply. Regards, Randy>>> Jay Krell 12/17/2008 3:47 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/17 15:47:34Modified files:cm3/scripts/python/: pylib.py make-dist.py Log message:fix line endings and 'temporarily' skip serial package -- there's a problem here on I386_CYGWIN I forgot about where essentially you want to provide both FilePosix and FileWin32; FilePosix will export File, FileWin32 will export FileWin32, so that portable Modula-3 can use File/FilePosix and Win32 specific serial code can use FileWin32; maybe some day we should try out this serial stuff, and /maybe/ the Posix paths work on Cygwin? For now, just skip it. And fix line endings. And try out the alternate 'clean' path in make-dist -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 17 23:52:14 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 22:52:14 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some. Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable" You throw this word around with emphasis a lot. Portable Modula-3 code. The lie of portable C code. What do you mean? There are many ways to write portable and non-portable Modula-3 and C code. For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*. It's quite easy to do either way, in either language. I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile. They are both actually very portable, but less portable than calling libraries that wrap them. What I mean is, Posix open is implemented on tons of systems, but not all. Win32 CreateFile is also implemented on tons of systems, but not all. It depends partly how you quantify systems. Win32 Create 1) is implemented on over a billion actual machines. open is actually in msvcrt.dll also, but let's ignore it for this discussion. I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever. I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. (Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything. I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library. It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C. If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix. I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting. That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable. Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained. As well, there is OpenMP. Another angle actually you can play here is quality of compiler. I did some experimentation with a range of compilers I had..I guess I was using C++ at the time. Some older compilers really stunk. Like, with CFront, you couldn't have more than one return. Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation. Cheating I think. See also Python and Perl. Well, ok, Modula-3 wins here by virtue of having a good small language spec. You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Dec 18 00:13:12 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 17 Dec 2008 23:13:12 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: ps: similar question: What does "low level" mean? hand.c and dtoa.c seem low level to me. I know, they are excused because they are rare and in m3core. FilePosixC.c does not. The stuff I did for stat, somewhat. The garbage collector does. But I can't define it, other than, uselessly "dealing with bits is low level", but int is_even(int a) { return !(a & 1); } doesn't seem very "low level", just slightly, and is very "portable". I was given a book about "complexity" as a gift. It makes a point that "complex" and "simple" are hard to define. Many things that seem the one, also seem the other. I have to say, I have whittled down unix/*.i3 a bunch, for all the "new" platforms, and I look at what is left, and I am inclined to keep "attacking" it, keep making it "smaller", like the bitflags in Ustat.i3. I am inclined to write something like this: Ustat.i3: TYPE StatFlags = RECORD X, R, W: INTEGER; (* maybe u_short *) END; PROCEDURE GetStatFlags(VAR StatFlags); UstatC.c typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; const static StatFlags_t StatsFlags = { X, R, W }; void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; Any module using any of the flags, would retrieve them in initialization. A bit inefficient. Portable. Cuts out many repeated lines. The C could would need a bit of #ifdefing, to handle platforms that don't have some of the flags, to make them 0. Again, very portable. Cuts out repetition. Is a little slower. I know it also might not work, like if there are switch statements or code needing actual constant constants. - Jay From: jay.krell at cornell.eduTo: hosking at cs.purdue.eduCC: m3devel at elegosoft.comSubject: RE: [M3devel] Fwd: Output from "cron" commandDate: Wed, 17 Dec 2008 22:52:14 +0000 1) I totally don't mind the emails. I can ignore them for a day or so easily. :) 2) C code in m3core or elsewhere. This is a philisophical question I couldn't decide, interested in discussing some.Originally I put "FilePosixC.c" in m3core, and called it from libm3. But I think that's..not great. 3) "portable"You throw this word around with emphasis a lot.Portable Modula-3 code.The lie of portable C code.What do you mean? There are many ways to write portable and non-portable Modula-3 and C code.For example there were Modula-3 regression tests printing the size of an integer. Portable to lots of systems, but not all. I can write Modula-3 code or C code dependent or independent of sizeof(int) or long or void*.It's quite easy to do either way, in either language.I don't think that proves anything. Likewise, in both Modula-3 and C, I can easily directly call Posix open, or Win32 CreateFile.They are both actually very portable, but less portable than calling libraries that wrap them.What I mean is, Posix open is implemented on tons of systems, but not all.Win32 CreateFile is also implemented on tons of systems, but not all.It depends partly how you quantify systems.Win32 Create 1) is implemented on over a billion actual machines.open is actually in msvcrt.dll also, but let's ignore it for this discussion.I've read that the "Unix workstation" market is only like in the single digit millions, either sold per year or sold ever.I realize those billion PCs that run Windows, can also run *BSD and Linux, making this all gray. Now, another way to quantify this though, that I find "impressive", is that Win32 CreateFile at least historically was implemented on PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad.(Basically, NT has been ported among the most of any /commercial closed source/ OS.) But again, I don't think this proves anything.I can pretty equally easily write portable or non-portable C or Modula, "at the library level". One angle you might argue, is that Modula-3 has a "built in" portability layer/library.It is, in a sense, easier to not call CreateFile or open in Modula-3 than it is in C.If that is what you mean, ok. HOWEVER, if you look at the Modula-3 system, the portability layer, you know, it is forked in terms of Posix and Win32. If you limit yourself to the Posix part of it, well, then, you can use Posix.I realize Posix systems are not all equal, they are all supersets of Posix, with some overlap in their supersetting.That's why my checkin worked on some systems but not others. Also, with respect to I/O, if use just stdio -- very portable.Threading -- finally, soon, multi threading will be very portable in C++, at least to the implementations still being maintained.As well, there is OpenMP. Another angle actually you can play here is quality of compiler.I did some experimentation with a range of compilers I had..I guess I was using C++ at the time.Some older compilers really stunk. Like, with CFront, you couldn't have more than one return.Therefore, code like: int even(int a) { if (a & 1) return 0; return 1; } /could/ be deemed not portable. Modula-3 wins here by virtue of there being only one implementation.Cheating I think. See also Python and Perl.Well, ok, Modula-3 wins here by virtue of having a good small language spec.You know..Perl is speced as the language implemented by perl.exe....so big and gnarly and impossible to predict..impossible to write down...C++ is fairly well speced, but only through mountains of paper and endless discussion and clarifications.... - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output from "cron" command No, broken briefly then fixed quickly is OK -- that's what regressions are for. But on the other hand, testing before checkin is probably nicer to the rest of the world. That way you don't get whinging e-mails from me complaining that the Tinderbox regressions broke. I suppose it depends which pain you prefer: my e-mails or testing comprehensively. ;-) I would hope that these sorts of system-dependent changes are minimized by keeping as much code in *portable* Modula-3 as possible. To my mind, nothing outside of m3core should need to escape to C. On 18 Dec 2008, at 09:24, Jay wrote: (critical typo -- you can provide a time for it to power back on.)I'm loathe to leave all my machines on and burn the electricity..I don't trust them to have good power management, lower power when idle.I should automate something here though, see if I can run Tinderbox, and see if I can get cron to power on/off.Every time I have looked at the Tinderbox it seemed too difficult to run. On the other hand, I don't know if the status quo is so bad.You tell me it's broken. I understand that is not ideal, and I should fix it fairly asap, but is it terrible every so often?I do tend to at least build multiple platforms, even if not "all"..so not every checkin breaks anything/everything. I figure LINUXLIBC6 is the most popular, and I can always test that on birch. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] Fwd: Output from "cron" commandDate: Thu, 18 Dec 2008 09:12:52 +1100 My machine is in a machine room at Purdue, on all the time. On 18 Dec 2008, at 08:37, Jay wrote: I keep most of my machines powered off most of the time. Do you have automation that both runs a daily Tinderbox, and turns the machine on/off?My SGI machine at least has a nice feature where you can software power it off, and provide a time that it should power down. I haven't automated, but it seems ideal for daily Tinderboxes. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu, 18 Dec 2008 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, I am still getting a compile error on whatever changes you made... -- Tony Begin forwarded message: From: Tony Hosking Date: 18 December 2008 00:04:32 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 2008.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return value: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN = 0--2008.12.17 06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt---checkout, compile and test of cm3 ...2008.12.17 06:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008.12.17 07:16:18]CHECKOUT_RETURN = 0--2008.12.17 07:16:21 -- compile in progress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compile 2008.12.17 08:01:31]COMPILE_RETURN = 02008.12.17 08:01:38 -- tests in progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-065704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN = 02008.12.17 08:01:38 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Dec 18 00:38:06 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 18 Dec 2008 10:38:06 +1100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812171304.mBHD4WO8009284@niagara.cs.purdue.edu> <54771545-2F85-42AE-AC4C-DF519AD2935D@cs.purdue.edu> <668CF277-E0D9-4BB9-89C2-013682CCBC17@cs.purdue.edu> Message-ID: <292A034D-ED30-407E-B773-F7DB513524EF@cs.purdue.edu> Think about the difference between safe and UNSAFE in Modula-3. On 18 Dec 2008, at 10:13, Jay wrote: > ps: similar question: > > What does "low level" mean? > > hand.c and dtoa.c seem low level to me. > I know, they are excused because they are rare and in m3core. > > FilePosixC.c does not. > The stuff I did for stat, somewhat. > > The garbage collector does. > But I can't define it, other than, uselessly "dealing with bits is > low level", but > int is_even(int a) { return !(a & 1); } doesn't seem very "low > level", just slightly, and is very "portable". > > I was given a book about "complexity" as a gift. > It makes a point that "complex" and "simple" are hard to define. > Many things that seem the one, also seem the other. > > I have to say, I have whittled down unix/*.i3 a bunch, for all the > "new" platforms, and I look at what is left, and I am inclined to > keep "attacking" it, keep making it "smaller", like the bitflags in > Ustat.i3. > > I am inclined to write something like this: > > Ustat.i3: > > TYPE StatFlags = RECORD > X, R, W: INTEGER; (* maybe u_short *) > END; > > PROCEDURE GetStatFlags(VAR StatFlags); > > UstatC.c > > typedef struct { long /* maybe u_short */ X, R, W; } StatFlags_t; > const static StatFlags_t StatsFlags = { X, R, W }; > void GetStatFlags(StatFlags_t* Result) { *Result = StatFlags }; > > Any module using any of the flags, would retrieve them in > initialization. > A bit inefficient. > Portable. > Cuts out many repeated lines. > > The C could would need a bit of #ifdefing, to handle platforms that > don't have some of the flags, to make them 0. > > Again, very portable. > Cuts out repetition. > Is a little slower. > I know it also might not work, like if there are switch statements > or code needing actual constant constants. > > > - Jay > > > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] Fwd: Output from "cron" command > Date: Wed, 17 Dec 2008 22:52:14 +0000 > > 1) I totally don't mind the emails. I can ignore them for a day or > so easily. :) > > > 2) C code in m3core or elsewhere. This is a philisophical question I > couldn't decide, interested in discussing some. > Originally I put "FilePosixC.c" in m3core, and called it from libm3. > But I think that's..not great. > > > 3) "portable" > You throw this word around with emphasis a lot. > Portable Modula-3 code. > The lie of portable C code. > What do you mean? > > > There are many ways to write portable and non-portable Modula-3 and > C code. > For example there were Modula-3 regression tests printing the size > of an integer. > Portable to lots of systems, but not all. > > I can write Modula-3 code or C code dependent or independent of > sizeof(int) or long or void*. > It's quite easy to do either way, in either language. > I don't think that proves anything. > > > Likewise, in both Modula-3 and C, I can easily directly call Posix > open, or Win32 CreateFile. > They are both actually very portable, but less portable than calling > libraries that wrap them. > What I mean is, Posix open is implemented on tons of systems, but > not all. > Win32 CreateFile is also implemented on tons of systems, but not all. > It depends partly how you quantify systems. > Win32 Create 1) is implemented on over a billion actual machines. > open is actually in msvcrt.dll also, but let's ignore it for this > discussion. > I've read that the "Unix workstation" market is only like in the > single digit millions, either sold per year or sold ever. > I realize those billion PCs that run Windows, can also run *BSD and > Linux, making this all gray. > > Now, another way to quantify this though, that I find "impressive", > is that Win32 CreateFile at least historically was implemented on > PowerPC, MIPS, Alpha, and currently x86, IA64, AMD64. Not bad. > (Basically, NT has been ported among the most of any /commercial > closed source/ OS.) > > But again, I don't think this proves anything. > I can pretty equally easily write portable or non-portable C or > Modula, "at the library level". > > One angle you might argue, is that Modula-3 has a "built in" > portability layer/library. > It is, in a sense, easier to not call CreateFile or open in Modula-3 > than it is in C. > If that is what you mean, ok. > > HOWEVER, if you look at the Modula-3 system, the portability layer, > you know, it is forked in terms of Posix and Win32. If you limit > yourself to the Posix part of it, well, then, you can use Posix. > I realize Posix systems are not all equal, they are all supersets of > Posix, with some overlap in their supersetting. > That's why my checkin worked on some systems but not others. > > Also, with respect to I/O, if use just stdio -- very portable. > Threading -- finally, soon, multi threading will be very portable in > C++, at least to the implementations still being maintained. > As well, there is OpenMP. > > Another angle actually you can play here is quality of compiler. > I did some experimentation with a range of compilers I had..I guess > I was using C++ at the time. > Some older compilers really stunk. Like, with CFront, you couldn't > have more than one return. > Therefore, code like: > int even(int a) > { > if (a & 1) > return 0; > return 1; > } > > /could/ be deemed not portable. > > Modula-3 wins here by virtue of there being only one implementation. > Cheating I think. > > See also Python and Perl. > Well, ok, Modula-3 wins here by virtue of having a good small > language spec. > You know..Perl is speced as the language implemented by > perl.exe....so big and gnarly and impossible to predict..impossible > to write down...C++ is fairly well speced, but only through > mountains of paper and endless discussion and clarifications.... > > > - Jay > > > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 09:27:36 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Fwd: Output from "cron" command > > > > No, broken briefly then fixed quickly is OK -- that's what > regressions are for. But on the other hand, testing before checkin > is probably nicer to the rest of the world. That way you don't get > whinging e-mails from me complaining that the Tinderbox regressions > broke. I suppose it depends which pain you prefer: my e-mails or > testing comprehensively. ;-) > > I would hope that these sorts of system-dependent changes are > minimized by keeping as much code in *portable* Modula-3 as > possible. To my mind, nothing outside of m3core should need to > escape to C. > > On 18 Dec 2008, at 09:24, Jay wrote: > > (critical typo -- you can provide a time for it to power back on.) > I'm loathe to leave all my machines on and burn the electricity.. > I don't trust them to have good power management, lower power when > idle. > I should automate something here though, see if I can run Tinderbox, > and see if I can get cron to power on/off. > Every time I have looked at the Tinderbox it seemed too difficult to > run. > > On the other hand, I don't know if the status quo is so bad. > You tell me it's broken. I understand that is not ideal, and I > should fix it fairly asap, but is it terrible every so often? > I do tend to at least build multiple platforms, even if not > "all"..so not every checkin breaks anything/everything. > > I figure LINUXLIBC6 is the most popular, and I can always test that > on birch. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3devel] Fwd: Output from "cron" command > Date: Thu, 18 Dec 2008 09:12:52 +1100 > > > My machine is in a machine room at Purdue, on all the time. > > On 18 Dec 2008, at 08:37, Jay wrote: > > I keep most of my machines powered off most of the time. > > Do you have automation that both runs a daily Tinderbox, and turns > the machine on/off? > My SGI machine at least has a nice feature where you can software > power it off, and provide a time that it should power down. I > haven't automated, but it seems ideal for daily Tinderboxes. > > - Jay > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Thu, 18 Dec 2008 07:10:05 +1100 > CC: m3devel at elegosoft.com > Subject: [M3devel] Fwd: Output from "cron" command > > > Jay, > > I am still getting a compile error on whatever changes you made... > > -- Tony > > Begin forwarded message: > > From: Tony Hosking > Date: 18 December 2008 00:04:32 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:30:07 -- checkout in progress. > [start checkout 2008.12.17 06:30:11] > cd /tmp/build-cm3-20081217-063006-fOaGeh/build > cvs return value: 0 > [end checkout 2008.12.17 06:49:43] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 06:49:45 -- compile in progress. > [start compile 2008.12.17 06:49:45] > compile return value: 0 > [end compile 2008.12.17 06:55:03] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081217-063006-fOaGeh ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-57-02 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081217-065704-RGaGCn/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.17 06:57:04 -- checkout in progress. > [start checkout 2008.12.17 06:57:06] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > cvs return value: 0 > [end checkout 2008.12.17 07:16:18] > CHECKOUT_RETURN = 0 > -- > 2008.12.17 07:16:21 -- compile in progress. > [start compile 2008.12.17 07:16:21] > compile return value: 0 > [end compile 2008.12.17 08:01:31] > COMPILE_RETURN = 0 > 2008.12.17 08:01:38 -- tests in progress. > [start run-tests 2008.12.17 08:01:38] > cd /tmp/build-cm3-20081217-065704-RGaGCn/build > [end run-tests 2008.12.17 08:01:38] > TESTS_RETURN = 0 > 2008.12.17 08:01:38 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081217-065704-RGaGCn ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 04:10:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:10:15 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." Message-ID: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- From mika at async.caltech.edu Thu Dec 18 04:33:46 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 17 Dec 2008 19:33:46 -0800 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: Your message of "Wed, 17 Dec 2008 22:52:14 GMT." Message-ID: <200812180333.mBI3XktJ079016@camembert.async.caltech.edu> I don't know what kind of a programming Superman you are. I switched from C to Modula-3 in 1999 because I was tired of having a new crop of bugs every time my C programs touched a new OS. Or, on Linux, every time there was a new version of Linux out (i.e., every day, twice on Sundays?). Yes, they were in fact flaws in my code. I don't doubt that there could exist bug-free portable programs coded in C that do what my programs were supposed to do. But in my experience, very very few programmers know C well enough to write truly portable programs. I'm also not one of them even though I've written more C than I care to remember. In Modula-3, it just seems natural...in fact it takes effort to write a program that *isn't* portable, the opposite of C. In fact, when I was teaching CS2 at Caltech, our CS1-2-3 sequence was sometimes known as the "Useless Languages Sequence" (it was supposed to be intro to programming, program construction/basic algorithms and data structures, and software engineering). CS1 was MIT's 6.001 (in Scheme); CS2 I taught in Modula-3; and someone taught CS3 in Eiffel. Each of the years I taught CS2 I had some students babbling on about how they wanted to study the material in C or C++, because they knew it and thought the other students would be more employable (it would be good for them). Well these self-professed C experts as good as never could answer some basic questions about C that have some very odd answers... For instance, if the following is a complete C program... int main() { int a[10]; int *b; b=... } how many values can b (legally) have? I believe the answer is... 13? Don Knuth once sent me a check for $2.56 because I spotted some non-conforming C in Vol. 2... If you don't know all those rules and think about them CONSTANTLY you will one day run into a compiler that makes you a very unhappy programmer. Sigh. I remember debugging a program that was comparing function pointers for ordering once... on a Convex. Yes the program was incorrect---unlike any other types of pointers, function pointers can only be compared for equality---so you might say we got what we deserved? (Of course the bloody thing did precisely what the programmer wanted on SunOS/sparc, NetBSD/i386, OSF/1 on the alpha, NetBSD on the hp300, solaris/sparc and every other OS we ever tried it on; and it didn't crash immediately nor emit warnings on the Convex but crashed sometimes minutes (millions of instructions) later.) Well @#%@#%%@ like that just doesn't happen in Modula-3. Mika Jay writes: >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :) >=20 >=20 >2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some. >Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great. >=20 >=20 >3) "portable" >You throw this word around with emphasis a lot. >Portable Modula-3 code. >The lie of portable C code. >What do you mean? >=20 >=20 >There are many ways to write portable and non-portable Modula-3 and C code. >For example there were Modula-3 regression tests printing the size of an in= >teger. > Portable to lots of systems=2C but not all. >=20 >I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*. >It's quite easy to do either way=2C in either language. >I don't think that proves anything. >=20 >=20 >Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile. >They are both actually very portable=2C but less portable than calling libr= >aries that wrap them. >What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll. >Win32 CreateFile is also implemented on tons of systems=2C but not all. >It depends partly how you quantify systems. >Win32 Create 1) is implemented on over a billion actual machines. >open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion. >I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever. >I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray. >=20 >Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently x86=2C IA64=2C AMD64. Not bad. >(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.) >=20 >But again=2C I don't think this proves anything. >I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level". >=20 >One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library. >It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C. >If that is what you mean=2C ok. >=20 >HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix. >I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting. >That's why my checkin worked on some systems but not others. >=20 >Also=2C with respect to I/O=2C if use just stdio -- very portable. >Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained. >As well=2C there is OpenMP. >=20 >Another angle actually you can play here is quality of compiler. >I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time. >Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return. >Therefore=2C code like: > int even(int a) =20 > { if (a & 1) return 0=3B =20 > return 1=3B =20 > } >=20 >/could/ be deemed not portable. >=20 >Modula-3 wins here by virtue of there being only one implementation. >Cheating I think. >=20 >See also Python and Perl. >Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec. >You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications.... >=20 >=20 > - Jay > > > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 09:27:36 +1100CC: m3devel at elegosoft.comSubject: Re: [M3devel] Fwd: Output= > from "cron" command > > >No=2C broken briefly then fixed quickly is OK -- that's what regressions ar= >e for. But on the other hand=2C testing before checkin is probably nicer t= >o the rest of the world. That way you don't get whinging e-mails from me c= >omplaining that the Tinderbox regressions broke. I suppose it depends whic= >h pain you prefer: my e-mails or testing comprehensively. =3B-) > >I would hope that these sorts of system-dependent changes are minimized by = >keeping as much code in *portable* Modula-3 as possible. To my mind=2C not= >hing outside of m3core should need to escape to C. > >On 18 Dec 2008=2C at 09:24=2C Jay wrote: > >(critical typo -- you can provide a time for it to power back on.)I'm loath= >e to leave all my machines on and burn the electricity..I don't trust them = >to have good power management=2C lower power when idle.I should automate so= >mething here though=2C see if I can run Tinderbox=2C and see if I can get c= >ron to power on/off.Every time I have looked at the Tinderbox it seemed too= > difficult to run. On the other hand=2C I don't know if the status quo is s= >o bad.You tell me it's broken. I understand that is not ideal=2C and I shou= >ld fix it fairly asap=2C but is it terrible every so often?I do tend to at = >least build multiple platforms=2C even if not "all"..so not every checkin b= >reaks anything/everything. I figure LINUXLIBC6 is the most popular=2C and I= > can always test that on birch. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] = >Fwd: Output from "cron" commandDate: Thu=2C 18 Dec 2008 09:12:52 +1100 > > >My machine is in a machine room at Purdue=2C on all the time. > >On 18 Dec 2008=2C at 08:37=2C Jay wrote: > >I keep most of my machines powered off most of the time. Do you have automa= >tion that both runs a daily Tinderbox=2C and turns the machine on/off?My SG= >I machine at least has a nice feature where you can software power it off= >=2C and provide a time that it should power down. I haven't automated=2C bu= >t it seems ideal for daily Tinderboxes. - Jay > >From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Thu=2C 18 Dec 200= >8 07:10:05 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output fro= >m "cron" command > > >Jay=2C > >I am still getting a compile error on whatever changes you made... > >-- Tony > >Begin forwarded message: > > >From: Tony Hosking >Date: 18 December 2008 00:04:32 GMT+11:00 >To: hosking at cs.purdue.edu >Subject: Output from "cron" command > >Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.s= >hproduced the following output:TESTHOSTNAME=3DniagaraWS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-30-04LASTREL=3D5.4.0INSTROOT_REL=3D/homes/h= >osking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm= >3-inst/niagara/prev-okINSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/l= >ast-okINSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE= >=3DPOSIXCM3_TARGET=3DSOLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-S= >OLgnu-5.4.0.tgzCM3CVSSERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegoso= >ft.com:/usr/cvsBINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.eleg= >osoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Bu= >ildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file = >/tmp/build-cm3-20081217-063006-fOaGeh/log.txt---checkout=2C compile and tes= >t of cm3 ...2008.12.17 06:30:07 -- checkout in progress.[start checkout 200= >8.12.17 06:30:11]cd /tmp/build-cm3-20081217-063006-fOaGeh/buildcvs return v= >alue: 0[end checkout 2008.12.17 06:49:43]CHECKOUT_RETURN =3D 0--2008.12.17 = >06:49:45 -- compile in progress.[start compile 2008.12.17 06:49:45]compile = >return value: 0[end compile 2008.12.17 06:55:03]COMPILE_RETURN =3D 1*** COM= >PILE FAILEDremoving build tree /tmp/build-cm3-20081217-063006-fOaGeh ...cle= >aning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regress= >ion test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3te= >st log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hoskin= >g/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.= >stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-= >min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLg= >nu-*.htmlTESTHOSTNAME=3DniagaraWS=3D/homes/hosking/work/cm3-ws/niagara-2008= >-12-17-11-57-02LASTREL=3D5.4.0INSTROOT_REL=3D/homes/hosking/work/cm3-inst/n= >iagara/rel-5.4.0INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok= >INSTROOT_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=3D/= >homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=3DPOSIXCM3_TARGET=3DS= >OLgnuBINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSS= >ERVER=3Dbirch.elegosoft.comCM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvsBINDIST= >MIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=3D/homes/hosking/work/c= >m3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.ele= >gosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu = >SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081217-0= >65704-RGaGCn/log.txt---checkout=2C compile and test of cm3 ...2008.12.17 06= >:57:04 -- checkout in progress.[start checkout 2008.12.17 06:57:06]cd /tmp/= >build-cm3-20081217-065704-RGaGCn/buildcvs return value: 0[end checkout 2008= >.12.17 07:16:18]CHECKOUT_RETURN =3D 0--2008.12.17 07:16:21 -- compile in pr= >ogress.[start compile 2008.12.17 07:16:21]compile return value: 0[end compi= >le 2008.12.17 08:01:31]COMPILE_RETURN =3D 02008.12.17 08:01:38 -- tests in = >progress.[start run-tests 2008.12.17 08:01:38]cd /tmp/build-cm3-20081217-06= >5704-RGaGCn/build[end run-tests 2008.12.17 08:01:38]TESTS_RETURN =3D 02008.= >12.17 08:01:38 -- checkout=2C compile and test run done.---removing build t= >ree /tmp/build-cm3-20081217-065704-RGaGCn ...cleaning CM3 workspaces.../hom= >es/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes= >/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hoski= >ng/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*= >.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snap= >shot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcle= >aning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone.= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >1) I totally don't mind the emails. I can ignore them for a day or so easil= >y. :)
> =3B
> =3B
>2) C code in m3core or elsewhere. This is a philisophical question I couldn= >'t decide=2C interested in discussing some.
>Originally I put "FilePosixC.c" in m3core=2C and called it from libm3. But = >I think that's..not great.
> =3B
> =3B
>3) "portable"
>You throw this word around with emphasis a lot.
>Portable Modula-3 code.
>The lie of portable C code.
>What do you mean?
> =3B
> =3B
>There are many ways to write portable and non-portable Modula-3 and C code.= >
>For example there were Modula-3 regression tests printing the size of an in= >teger.
> =3B Portable to lots of systems=2C but not all.
> =3B
>I can write Modula-3 code or C code dependent or independent of sizeof(int)= > or long or void*.
>It's quite easy to do either way=2C in either language.
>I don't think that proves anything.
> =3B
> =3B
>Likewise=2C in both Modula-3 and C=2C I can easily directly call Posix open= >=2C or Win32 CreateFile.
>They are both actually very portable=2C but less portable than calling libr= >aries that wrap them.
>What I mean is=2C Posix open is implemented on tons of systems=2C but not a= >ll.
>Win32 CreateFile is also implemented on tons of systems=2C but not all.
>It depends partly how you quantify systems.
>Win32 Create 1) is implemented on over a billion actual machines.
>open is actually in msvcrt.dll also=2C but let's ignore it for this discuss= >ion.
>I've read that the "Unix workstation" market is only like in the single dig= >it millions=2C either sold per year or sold ever.
>I realize those billion PCs that run Windows=2C can also run *BSD and Linux= >=2C making this all gray.
> =3B
>Now=2C another way to quantify this though=2C that I find "impressive"=2C i= >s that Win32 CreateFile at least historically was implemented on PowerPC=2C= > MIPS=2C Alpha=2C and currently =3Bx86=2C IA64=2C AMD64. Not bad.
>(Basically=2C NT has been ported among the most of any /commercial closed s= >ource/ OS.)
> =3B
>But again=2C I don't think this proves anything.
>I can pretty equally easily write portable or non-portable C or Modula=2C "= >at the library level".
> =3B
>One angle you might argue=2C is that Modula-3 has a "built in" portability = >layer/library.
>It is=2C in a sense=2C easier to not call CreateFile or open in Modula-3 th= >an it is in C.
>If that is what you mean=2C ok.
> =3B
>HOWEVER=2C if you look at the Modula-3 system=2C the portability layer=2C y= >ou know=2C it is forked in terms of Posix and Win32. If you limit yourself = >to the Posix part of it=2C well=2C then=2C you can use Posix.
>I realize Posix systems are not all equal=2C they are all supersets of Posi= >x=2C with some overlap in their supersetting.
>That's why my checkin worked on some systems but not others.
> =3B
>Also=2C with respect to I/O=2C if use just stdio -- very portable.
>Threading -- finally=2C soon=2C multi threading will be very portable in C+= >+=2C at least to the implementations still being maintained.
>As well=2C there is OpenMP.
> =3B
>Another angle actually you can play here is quality of compiler.
>I did some experimentation with a range of compilers I had..I guess I was u= >sing C++ at the time.
>Some older compilers really stunk. Like=2C with CFront=2C you couldn't have= > more than one return.
>Therefore=2C code like:
> =3B int even(int a) =3B
> =3B { =3B
 =3B  =3B if (a &=3B 1) =3B
 = >=3B  =3B =3B =3B return 0=3B =3B
> =3B  =3B return 1=3B =3B
> =3B }
> =3B
>/could/ be deemed not portable.
> =3B
>Modula-3 wins here by virtue of there being only one implementation.
>Cheating I think.
> =3B
>See also Python and Perl.
>Well=2C ok=2C Modula-3 wins here by virtue of having a good small language = >spec.
>You know..Perl is speced as the language implemented by perl.exe....so big = >and gnarly and impossible to predict..impossible to write down...C++ is fai= >rly well speced=2C but only through mountains of paper and endless discussi= >on and clarifications....
> =3B
> =3B
> =3B- Jay


> >
>
>From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Date: Thu=2C 18= > Dec 2008 09:27:36 +1100
CC: m3devel at elegosoft.com
Subject: Re: [M3de= >vel] Fwd: Output from "cron" command


>
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >
No=2C broken briefly then fixed quickly is OK -- that's what regressio= >ns are for.  =3BBut on the other hand=2C testing before checkin is prob= >ably nicer to the rest of the world.  =3BThat way you don't get whingin= >g e-mails from me complaining that the Tinderbox regressions broke.  = >=3BI suppose it depends which pain you prefer: my e-mails or testing compre= >hensively.  =3B=3B-)
>

>
I would hope that these sorts of system-dependent changes are minimize= >d by keeping as much code in *portable* Modula-3 as possible.  =3BTo my= > mind=2C nothing outside of m3core should need to escape to C.
=

>
>
On 18 Dec 2008=2C at 09:24=2C Jay wrote:

erchange-newline> >
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>(critical typo -- you can provide a time for it to power back on.)
I'm = >loathe to leave all my machines on and burn the electricity..
I don't tr= >ust them to have good power management=2C lower power when idle.
I shoul= >d automate something here though=2C see if I can run Tinderbox=2C and see i= >f I can get cron to power on/off.
Every time I have looked at the Tinder= >box it seemed too difficult to run.
 =3B
On the other hand=2C I d= >on't know if the status quo is so bad.
You tell me it's broken. I unders= >tand that is not ideal=2C and =3BI should fix it fairly asap=2C but is = >it terrible every so often?
I do tend to at least build multiple platfor= >ms=2C even if not "all"..so not every checkin breaks anything/everything.R> =3B
I figure LINUXLIBC6 is the most popular=2C and I can always t= >est that on birch.
 =3B
 =3B- Jay


>
>
From: =3Bailto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:EC_Apple-converted-space> =3B.edu">jay.krell at cornell.edu
Subject: Re: [M3devel] Fwd: Output from = >"cron" command
Date: Thu=2C 18 Dec 2008 09:12:52 +1100


>
T: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-= >INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLL= >APSE: separate"> >
tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica= >=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WH= >ITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"><= >SPAN class=3DEC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12p= >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate">-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb= >(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: n= >ormal=3B BORDER-COLLAPSE: separate">yle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B= > COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER= >-SPACING: normal=3B BORDER-COLLAPSE: separate">yle-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFO= >RM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: norma= >l=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >
My machine is in a machine room at Purdue=2C on all the time.
PAN>
> >
>
On 18 Dec 2008=2C at 08:37=2C Jay wrote:

interchange-newline> >
=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)= >=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BO= >RDER-COLLAPSE: separate"> >
na">I keep most of my machines powered off most of the time.
 =3B>Do you have automation that both runs a daily Tinderbox=2C and turns the m= >achine on/off?
My SGI machine at least has a nice feature where you can = >software power it off=2C and provide a time that it should power down. I ha= >ven't automated=2C but it seems ideal for daily Tinderboxes.
 =3B> =3B- Jay


>
>From: =3Bilto:hosking at cs.purdue.edu">hosking at cs.purdue.edu
To:C_EC_Apple-converted-space> =3Bll.edu">jay.krell at cornell.edu
Date: Thu=2C 18 Dec 2008 07:10:05 +110= >0
CC: =3B"mailto:m3devel at elegosoft.com">m3devel at elegosoft.com
Subject: [M3dev= >el] Fwd: Output from "cron" command


>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate"> >
n style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: non= >e=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LE= >TTER-SPACING: normal=3B BORDER-COLLAPSE: separate">pple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-= >TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE= >: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">s=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helv= >etica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate">FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TE= >XT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-C= >OLLAPSE: separate">ACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0= >=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: nor= >mal=3B BORDER-COLLAPSE: separate">tyle=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none= >=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LET= >TER-SPACING: normal=3B BORDER-COLLAPSE: separate">ple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-T= >RANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE:= > normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">=3DEC_EC_EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helve= >tica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px= >=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separ= >ate"> >
Jay=2C
>

>
I am still getting a compile error on whatever changes you made...V> >

>
-- Tony
<= >/DIV>
>

>
Begin forwarded message:

ewline> >
>
>
= >ica color=3D#000000 size=3D3>From:ace> =3Belvetica size=3D3>Tony Hosking <=3Bu">hosking at cs.purdue.edu>=3B
>
= >ica color=3D#000000 size=3D3>Date:ace> =3Belvetica size=3D3>18 December 2008 00:04:32 GMT+11:00
>
= >ica color=3D#000000 size=3D3>To:e> =3Bvetica size=3D3>hosking at cs.purdue.= >edu
>
= >ica color=3D#000000 size=3D3>Subject:-space> =3B=3DHelvetica size=3D3>Output from "cron" command
>
MARGIN-RIGHT: 0px">
>
Your "cron" job on niagara.cs.purdue.edu
$HOME/cm3/scripts/regressi= >on/cron.sh

produced the following output:

TESTHOSTNAME=3Dniag= >ara
WS=3D/homes/hosking/work/cm3-ws/niagara-2008-12-17-11-30-04
LASTR= >EL=3D5.4.0
INSTROOT_REL=3D/homes/hosking/work/cm3-inst/niagara/rel-5.4.0= >
INSTROOT_POK=3D/homes/hosking/work/cm3-inst/niagara/prev-ok
INSTROOT= >_LOK=3D/homes/hosking/work/cm3-inst/niagara/last-ok
INSTROOT_CUR=3D/home= >s/hosking/work/cm3-inst/niagara/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET= >=3DSOLgnu
BINDISTMIN=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tg= >z
CM3CVSSERVER=3Dbirch.elegosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com= >:/usr/cvs
BINDISTMIN_NAME=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN= >=3D/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.= >elegosoft.com..
ssh birch.elegosoft.com ok
Building cm3.
Tinderbox= > Tree:  =3B =3B"cm3"
Buildname:  =3B =3B =3B =3B= > =3B =3B =3B"SOLgnu SunOS 5.10 niagara release-build"

cr= >eating log file /tmp/build-cm3-20081217-063006-fOaGeh/log.txt

--->
checkout=2C compile and test of cm3 ...
2008.12.17 06:30:07 -- chec= >kout in progress.
[start checkout 2008.12.17 06:30:11]
cd /tmp/build-= >cm3-20081217-063006-fOaGeh/build
cvs return value: 0
[end checkout 20= 08.12.17 06:49:43]
CHECKOUT_RETURN =3D 0
--
2008.12.17 06:49:45 --= > compile in progress.
[start compile 2008.12.17 06:49:45]
compile ret= >urn value: 0
[end compile 2008.12.17 06:55:03]
COMPILE_RETURN =3D 1R>*** COMPILE FAILED
removing build tree /tmp/build-cm3-20081217-063006-= >fOaGeh ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/niag= >ara-*

cleaning regression test log files...
/homes/hosking/tmp/cm= >3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking/= >tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3te= >sts-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract= >

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-mi= >n-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-r= >eport-SOLgnu-*.html

TESTHOSTNAME=3Dniagara
WS=3D/homes/hosking/wo= >rk/cm3-ws/niagara-2008-12-17-11-57-02
LASTREL=3D5.4.0
INSTROOT_REL=3D= >/homes/hosking/work/cm3-inst/niagara/rel-5.4.0
INSTROOT_POK=3D/homes/hos= >king/work/cm3-inst/niagara/prev-ok
INSTROOT_LOK=3D/homes/hosking/work/cm= >3-inst/niagara/last-ok
INSTROOT_CUR=3D/homes/hosking/work/cm3-inst/niaga= >ra/current
CM3_OSTYPE=3DPOSIX
CM3_TARGET=3DSOLgnu
BINDISTMIN=3D/ho= >mes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
CM3CVSSERVER=3Dbirch.ele= >gosoft.com
CM3CVSROOT=3Dbirch.elegosoft.com:/usr/cvs
BINDISTMIN_NAME= >=3Dcm3-min-POSIX-SOLgnu-5.4.0.tgz
BINDISTMIN=3D/homes/hosking/work/cm3-m= >in-POSIX-SOLgnu-5.4.0.tgz
testing ssh birch.elegosoft.com..
ssh birch= >.elegosoft.com ok
Building cm3.
Tinderbox Tree:  =3B =3B"cm3"= >
Buildname:  =3B =3B =3B =3B =3B =3B =3B"SOL= >gnu SunOS 5.10 niagara lastok-build"

creating log file /tmp/build-cm= >3-20081217-065704-RGaGCn/log.txt

---

checkout=2C compile and = >test of cm3 ...
2008.12.17 06:57:04 -- checkout in progress.
[start c= >heckout 2008.12.17 06:57:06]
cd /tmp/build-cm3-20081217-065704-RGaGCn/bu= >ild
cvs return value: 0
[end checkout 2008.12.17 07:16:18]
CHECKOU= >T_RETURN =3D 0
--
2008.12.17 07:16:21 -- compile in progress.
[sta= >rt compile 2008.12.17 07:16:21]
compile return value: 0
[end compile = >2008.12.17 08:01:31]
COMPILE_RETURN =3D 0
2008.12.17 08:01:38 -- test= >s in progress.
[start run-tests 2008.12.17 08:01:38]
cd /tmp/build-cm= >3-20081217-065704-RGaGCn/build
[end run-tests 2008.12.17 08:01:38]
TE= >STS_RETURN =3D 0
2008.12.17 08:01:38 -- checkout=2C compile and test run= > done.

---

removing build tree /tmp/build-cm3-20081217-065704= >-RGaGCn ...
cleaning CM3 workspaces...
/homes/hosking/work/cm3-ws/nia= >gara-*

cleaning regression test log files...
/homes/hosking/tmp/c= >m3/niagara/cm3-rlog-*

cleaning m3test log files...
/homes/hosking= >/tmp/cm3/niagara/m3tests-*.stdout

/homes/hosking/tmp/cm3/niagara/m3t= >ests-*.stderr

/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extrac= >t

cleaning snapshot files...
/homes/hosking/tmp/cm3/niagara/cm3-m= >in-POSIX-SOLgnu-*-*.tgz

cleaning package reports...
/tmp/cm3-pkg-= >report-SOLgnu-*.html

done.

PAN>

V>

body> >= > >--_90bb03af-9f51-49d3-a3e0-dadf1fa47719_-- From jay.krell at cornell.edu Thu Dec 18 13:25:48 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 12:25:48 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: > Do Cygwin users really care about all that stuff? Could be that some do, some don't. > The point of using Modula-3 on Cygwin But is "Cygwin" the point? Would Interix or UWin suffice? Maybe. Or just accepting forward slashes and providing a 64bit longint? Maybe. And what does "Cygwin" mean?It is many things. Maybe the point is to use X Windows? Not likely imho, but maybe. I hadn't considered this at first myself, but someone (Olaf?) asked for it. Or pthreads? Again not likely, but maybe. Or LONGINT? Maybe. Or Posix paths? ie. form and run command lines from bash. This seems to be the most popular reason. The "run" part, not a problem, the "form" part somewhat related to the path issue. Or to use a "super" optimizing compiler instead of just a fast "moderately" optimizing compiler? Or to use something you have more source to? i.e. gcc/ld/newlib. Or to interop with gcc-specific code? You can use a mix of compilers and interop, Cygwin, Visual C++, and the integrated backend all use the same object file format, but maybe it is easier to use one toolset more, than to mix. Or to use forward slashes in paths? You can do that with regular NT386 now, as you always could with the kernel32.dll functions. "Posix paths" vs. "forward slashes" not entirely the same thing, but almost. Unfortunately Win32 is not consistent here, like the file open dialogs and I think shlwapi.dll don't accept forward slashes. Arguably bugs. Or to interop with open/read/close/stat? You could always already do that too. Those functions do all exist in msvcr*.dll. Or to interop with very Berkeley socket-specific code? Maybe. I think Windows sockets are very very similar to Berkely/Posix sockets, but not quite the same. For example, I think an "fd set" is an array of opaque pointers, not a bit set. Or to link (again, interop) with code calling fork? Or because you want a better cross-build story? That is, you can build/run the Cygwin runtime, compiler, and linker on any other host, whereas the NT386 linker is only runnable on NT386. Or because you are familiar with gdb? Or due to issues around freedom of beer or liberty? Visual C++ has varied in its free-beer-ness. What is the point of the MinGWin project? I think it is about using autotools and gcc, but not sure. And producing binaries with "more flexible licensing" (no cygwin1.dll) One reason I got this stuff working is becauseI saw it as a shortcut to 64bit LONGINT support. I find Cygwin largely "cumbersome".Just usings its command line is slow.Slow to edit. gcc is slow.But the cross build story is very good.You know, I use Cygwin so that Windows can target Modula-3 to other systems. You know, to build m3cg. MinGWin I think suffices, but I got the impression at some point that Cygwin was better maintained than MinGWin. There are many factors.Modula-3 was already usable by Cygwin users, just as Visual C++ already is. The Modula-3 libraries shield you from almost all of this, if you stick to them.If you don't interop with much C code, if don't need a 64bit LONGINT, if thefast integrated backend generates efficient enough code for you, etc.The "external" interface really seems largely the form of paths. ...I /think/ most of the point to Cygwin to a lot of people, at some point, was to have a free beer C/C++ compiler. But again, this is only some people, and there are now other options. Digital Mars and Open Watcom are also pretty good. Digital Mars doesn't come with source. Open Watcom does. eBay provides pretty cheap stuff too. C/C++ compilers have lost their value as other programming languages (Java, JavaScript, C#, Perl, Python) have risen and sometimes been free (all free-beer, somewhat free-freedom/GPL). Somewhat due to the very "safety" issues that Modula-3 solves. Modula-3 I think was way ahead of its time. The world has mostly caught up, though Modula-3 is still a fairly unique hybrid, in terms of compiling directly to native code, and offering optional safety within one language. (C# also has optional safety within one language). If you only want safety and can sacrifice perf and/or generality and/or native code.. But again, there are many factors, many characteristics that define "Cygwin". - Jay> To: jay.krell at cornell.edu> Date: Wed, 17 Dec 2008 19:10:15 -0800> From: mika at async.caltech.edu> CC: jkrell at elego.de; m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > Do Cygwin users really care about all that stuff?> > The point of using Modula-3 on Cygwin is to turn a yucky Windows> machine as close as possible into a friendly Unix environment...> not to use a bunch of Microsoft tools and libraries---I would have> thought that if you wanted to use Visual X-Y-Z then you wouldn't> be using Cygwin but would be using the native Windows environment?> > So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> occasionally use on Windows uses Windows threads and has its own> compiler back-end. But it's kind of cool too and since it's not> really distinguishable from user-level threads and gcc I just think> "great" and get on with it. In fact I'm not even sure it uses> the integrated back end now that I think about it.> > I think what I'm saying is that almost no Cygwin users are terribly> interested in Microsoft interfaces, and I would bet most Cygwin> users aren't even aware of the existence of most Microsoft tools.> They (we) want as little to do with Windows as possible....> > Mika> > Jay writes:> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/plain; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >From: TonyTo: jay> >I would have thought of CygWin as a POSIX platform that just happens to hav=> >e Windows underneath. Did we go round this roundabout before?> >> >> >> >> >> >It's more complicated than that.> >It isn't just one bit.> >There's a few factors=2C that people care more and less about.> > GUI library -- Win32 or X> > thread library -- pthreads or Win32 (or user threads)=20> > file system path representation -- forward slashes or backward slashes => > This seems to be what people care most about -- to see forward slashes.> > C compiler -- Visual C++ cl or Cygwin/GNU gcc> > linker -- Visual C++ link or Cygwin/GNU ld=20> > C runtime -- msvcr*.dll or cygwin1.dll=20> > integrated backend or not The integrated backend is way faster=2C but=> > doesn't yet support 64bit LONGINT.> > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/=> >pdb> >Many of these factors are independent of each other=2C but not entirely.> >Compiler/linker directly lead to which debugger you can use=2C since they => >each use different formats.> >Well=2C you can use either debugger=2C but..without symbols..at least for s=> >ome modules.> >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa=> >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. => >This is probably fixable at least in one combination withan alias (ability => >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN=> >U ld understands aliases).> >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the=> >ir linker somehow accepts them.> >=20> >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by=> >te structs by value to __stdcall functions=2C the function name isn't mangl=> >ed correctly and linking fails.> >It is a bit of a combinatorial explosion.As the system stands=2C you can co=> >ntrol each bit in the config file=2C andmultiple combinations might work=2C=> > though I don't run any combinatorial tests.> >Three combinations are "identified" (named=2C spoken of) and have had some => >development testing.NT386=2C NT386MINGNU (which I think should be called I3=> >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN).> >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that=> >'s the name).Interix and UWIN being two other very viable Posix layers on W=> >indows.Interix has been renamed a few times. It is called "SFU" -- services=> > for Unix=2C and I don't know=2Clike SUA -- services for unix applications=> >=2C and I think there's another name.It is a free download from Microsoft f=> >or x86 Windows < Vista=2C and is an optional component in Vista.It has its => >pluses and minuses..not now.> >Anyway=2C in conversation here=2C one of the main things that people DO NOT=> > care aboutis which thread library is used.> >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h=> >ave used Win32 threads.> >And maybe never had any GUI?> >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres=> >h go at debugging it would pan out..but I'm really not too interested..> >I also have no interest in user threads. I believe they are never warranted=> >=2C excepton old systems that don't have kernel threads=2C like the hypothe=> >tical DJGPP port.> >I never brought up the issue of serial ports.I ASSUMED that almost nobody u=> >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem=> >ented using Cygwin or Win32=2C as long as they work.> >I ASSUME there is no interop issue=2C like getting an int fd from the seria=> >l portand passing it to select() or close=2C or getting a Win32 HANDLE and => >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly.> >I didn't look into if Cygwin is complete enough to implement them.I conside=> >r serial ports to be kind of out on the fringe..that it wouldn't be surpris=> >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui=> >lding minicom on some non-Linux system(s) recently and it failed miserably.=> > I don't know if there is even much Posix portability option for serial por=> >t programming.> >Sure..I debug over a serial port=2C and lately have machines I can't get vi=> >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us=> >e a serial consolefor initial install (recent achievement :) still haven't => >installed over network..)=2C but that's still imho a big step away from wri=> >ting code that uses the serial port.(And NetBSD doesn't support the local v=> >ideo on SGI O2=2C but I'm running OpenBSD there for now.)> >=20> >Anyway=2C there are a few ways to look at this issue.More and less trivial.> >A trivial view is: Get the filtering to work. Build serial on all platform=> >s but Cygwin. Or see if the Posix code will work.> > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p=> >rogrammers is not 0=2C but could be. I actually don't think anyone is using=> > Cygwin x Modula-3.> >Another view is: Get the Win32 serial code to work on Cygwin. The code is=> > already there=2C and buildable=2C but building in a system that also build=> >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio=> >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr=> >ovide the File interface=2C and FileWin32.m3 to provide just its own spec=> >ific FileWin32 interface. I think this is a simple transform of the code.> >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma=> >ybe I'll get the truncation I deserve. :)> > - Jay=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >Content-Type: text/html; charset="iso-8859-1"> >Content-Transfer-Encoding: quoted-printable> >> >> >> >> >> >> >
> >From: Tony
To: jay

> >
I would have thought of CygWin as a POSIX platform that just happens t=> >o have Windows underneath.  =3BDid we go round this roundabout before?<=> >/DIV>> >
>12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND=> >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS=> >E: separate">> >
>e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C=> >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S=> >PACING: normal=3B BORDER-COLLAPSE: separate"> >pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n=> >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B => >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN=> >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no=> >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T=> >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S=> >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate"> >class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet=> >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B=> > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate=> >"> >x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT=> >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: => >separate"> >ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX=> >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO=> >LLAPSE: separate"> >0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0=> >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B=> >ORDER-COLLAPSE: separate">> >

 >V>
> >
> >
> FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T=> >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-=> >COLLAPSE: separate">> >
>>
It's more complicated than that.
> >
>>
It isn't just one bit.
> >
>>
There's a few factors=2C that people care more and less about.
> >
>>
 =3B GUI library =3B -- Win32 or X
> >
>>
 =3B thread library =3B -- pthreads or Win32 (or user threads)=> >
> >
>>
 =3B file system path representation -- forward slashes or backwar=> >d slashes
 =3B =3B =3B This seems to be what people care mos=> >t about -- to see forward slashes.
> >
>>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
> >
>>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
> >
>>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
> >
>>
 =3B integrated backend or not
 =3B =3B =3B The in=> >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT. >IV>> >
>>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi=> >suaStudio/CodeView/pdb
> >
>>
Many of these factors are independent of each other=2C but not entirel=> >y.
> >
>>
Compiler/linker directly lead to which debugger you can use=2C since >R> =3B they each use different formats.
> >
>>
Well=2C you can use either debugger=2C but..without symbols..at least => >for some modules.
> >
>>
The Cygwin C runtime and Visual C++ linker disagree
on the name of => >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual=> > C++ linker. This is probably fixable at least in one combination with
a=> >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't=> >
know if Cygwin/GNU ld understands aliases).
> >
>>
I have also found that Cygwin produces incorrect
import .libs=2C pe=> >rhaps their linker somehow accepts them.
> >
>> =3B
> >
>>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b=> >yte structs by value to __stdcall functions=2C the function name isn't mang=> >led correctly and linking fails.
> >
>>
It is a bit of a combinatorial explosion.
As the system stands=2C y=> >ou can control each bit in the config file=2C and
multiple combinations => >might work=2C though I don't run any combinatorial tests.
> >
>>
Three combinations are "identified" (named=2C spoken of) =3Band ha=> >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho=> >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called=> > I386_CYGWIN).
> >
>>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if=> > that's the name).
Interix and UWIN being two other very viable Posix la=> >yers on Windows.
Interix has been renamed a few times. It is called "SFU=> >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u=> >nix applications=2C and I think there's another name.
It is a free downl=> >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo=> >nent in Vista.
It has its pluses and minuses..not now.
> >
>>
Anyway=2C in conversation here=2C one of the main things that people D=> >O NOT care about
is which thread library is used.
> >
>>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi=> >ght have used Win32 threads.
> >
>>And maybe never had any GUI?
> >
>>
I tried getting pthreads to work=2C but gave up debugging it.
Perha=> >ps a fresh go at debugging it would pan out..but I'm really not too interes=> >ted..
> >
>>
I also have no interest in user threads. I believe they are never warr=> >anted=2C except
on old systems that don't have kernel threads=2C like th=> >e hypothetical DJGPP port.
> >
>>
I never brought up the issue of serial ports.
I ASSUMED that almost=> > nobody uses them=2C and anyone that does use them=2C doesn't
care if th=> >ey are implemented using Cygwin or Win32=2C as long as they work.
> >
>>I ASSUME there is no interop issue=2C like getting an int fd from the seri=> >al port
and passing it to select() or close=2C or getting a Win32 HANDLE=> > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c=> >ertainly.
> >
>>
I didn't look into if Cygwin is complete enough to implement them.
=> >I consider serial ports to be kind of out on the fringe..that it wouldn't b=> >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I => >tried building minicom on some non-Linux system(s) recently and it failed m=> >iserably. I don't know if there is even much Posix portability option for s=> >erial port programming.
> >
>>
Sure..I debug over a serial port=2C and lately have machines I can't g=> >et video
out of (Itanium) or even seem to have no video hardware (Sun)=> >=2C so I use a serial console
for initial install (recent achievement :)=> > still haven't installed over network..)=2C but that's still imho a big ste=> >p away from writing code that uses the serial port.
(And NetBSD doesn't => >support the local video on SGI O2=2C but I'm running OpenBSD there for now.=> >)
> >
>> =3B
> >
>>
Anyway=2C there are a few ways to look at this issue.
More and less=> > trivial.
> >
>>
A trivial view is:
 =3B Get the filtering to work. Build serial=> > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code => >will work.
> >
>> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria=> >l port programmers is not 0=2C but could be. I actually don't think anyone => >is using Cygwin x Modula-3.
> >
>>
Another view is:
 =3B Get the Win32 serial code to work on Cygw=> >in.
 =3B The code is already there=2C and buildable=2C but building => >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula=> >-3 rules=2C you end up with two implementations of the File interface.
&=> >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte=> >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci=> >fic FileWin32 interface.
 =3B I think this is a simple transform of => >the code.
> >
>>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor=> >ry.
Maybe I'll get the truncation I deserve. :)
> >
>>
 =3B- Jay
> >=> >> >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Thu Dec 18 21:46:26 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 18 Dec 2008 15:46:26 -0500 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> References: Your message of "Wed, 17 Dec 2008 22:05:30 GMT." <200812180310.mBI3AFNX078630@camembert.async.caltech.edu> Message-ID: <494A6FFD.1E75.00D7.1@scires.com> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel about it and Modula-3. Mika has a point that if you want to create an environment on Windows that is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are looking for. On the other hand, I am in a camp of folks who don't mind using the Microsoft tools and want Modula-3 to work out of the box on Windows without having to add any other non-Microsoft stuff (like Cygwin). I think it is great for CM3 to be able to support both these camps! Thus, you will find me always advocating for keeping the "CM3 on Windows variant" where it will work on Windows using just the Microsoft tools. I am happy with the use of Windows threads and the native backend and use of the MS Visual Studio linker. Regards, Randy Coleburn >>> Mika Nystrom 12/17/2008 10:10 PM >>> Do Cygwin users really care about all that stuff? The point of using Modula-3 on Cygwin is to turn a yucky Windows machine as close as possible into a friendly Unix environment... not to use a bunch of Microsoft tools and libraries---I would have thought that if you wanted to use Visual X-Y-Z then you wouldn't be using Cygwin but would be using the native Windows environment? So yes I find it a bit odd that the ancient PM3-Klagenfurt I still occasionally use on Windows uses Windows threads and has its own compiler back-end. But it's kind of cool too and since it's not really distinguishable from user-level threads and gcc I just think "great" and get on with it. In fact I'm not even sure it uses the integrated back end now that I think about it. I think what I'm saying is that almost no Cygwin users are terribly interested in Microsoft interfaces, and I would bet most Cygwin users aren't even aware of the existence of most Microsoft tools. They (we) want as little to do with Windows as possible.... Mika Jay writes: >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > >From: TonyTo: jay >I would have thought of CygWin as a POSIX platform that just happens to hav= >e Windows underneath. Did we go round this roundabout before? > > > > > >It's more complicated than that. >It isn't just one bit. >There's a few factors=2C that people care more and less about. > GUI library -- Win32 or X > thread library -- pthreads or Win32 (or user threads)=20 > file system path representation -- forward slashes or backward slashes = > This seems to be what people care most about -- to see forward slashes. > C compiler -- Visual C++ cl or Cygwin/GNU gcc > linker -- Visual C++ link or Cygwin/GNU ld=20 > C runtime -- msvcr*.dll or cygwin1.dll=20 > integrated backend or not The integrated backend is way faster=2C but= > doesn't yet support 64bit LONGINT. > which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView/= >pdb >Many of these factors are independent of each other=2C but not entirely. >Compiler/linker directly lead to which debugger you can use=2C since they = >each use different formats. >Well=2C you can use either debugger=2C but..without symbols..at least for s= >ome modules. >The Cygwin C runtime and Visual C++ linker disagreeon the name of __ImageBa= >se=2C so you can't link to the Cygwin C runtime with theVisual C++ linker. = >This is probably fixable at least in one combination withan alias (ability = >to link Cygwin C runtime with Visual C++ linker=3B I don'tknow if Cygwin/GN= >U ld understands aliases). >I have also found that Cygwin produces incorrectimport .libs=2C perhaps the= >ir linker somehow accepts them. >=20 >Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 by= >te structs by value to __stdcall functions=2C the function name isn't mangl= >ed correctly and linking fails. >It is a bit of a combinatorial explosion.As the system stands=2C you can co= >ntrol each bit in the config file=2C andmultiple combinations might work=2C= > though I don't run any combinatorial tests. >Three combinations are "identified" (named=2C spoken of) and have had some = >development testing.NT386=2C NT386MINGNU (which I think should be called I3= >86_MINGWIN)=2C and NT386GNU (which I think should be called I386_CYGWIN). >Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if that= >'s the name).Interix and UWIN being two other very viable Posix layers on W= >indows.Interix has been renamed a few times. It is called "SFU" -- services= > for Unix=2C and I don't know=2Clike SUA -- services for unix applications= >=2C and I think there's another name.It is a free download from Microsoft f= >or x86 Windows < Vista=2C and is an optional component in Vista.It has its = >pluses and minuses..not now. >Anyway=2C in conversation here=2C one of the main things that people DO NOT= > care aboutis which thread library is used. >I believe it was asserted that PM3/NT386GNU did work=2C but that it might h= >ave used Win32 threads. >And maybe never had any GUI? >I tried getting pthreads to work=2C but gave up debugging it.Perhaps a fres= >h go at debugging it would pan out..but I'm really not too interested.. >I also have no interest in user threads. I believe they are never warranted= >=2C excepton old systems that don't have kernel threads=2C like the hypothe= >tical DJGPP port. >I never brought up the issue of serial ports.I ASSUMED that almost nobody u= >ses them=2C and anyone that does use them=2C doesn'tcare if they are implem= >ented using Cygwin or Win32=2C as long as they work. >I ASSUME there is no interop issue=2C like getting an int fd from the seria= >l portand passing it to select() or close=2C or getting a Win32 HANDLE and = >passing itto WaitForSingleObject=2C but I could be wrong here=2C certainly. >I didn't look into if Cygwin is complete enough to implement them.I conside= >r serial ports to be kind of out on the fringe..that it wouldn't be surpris= >ing if Cygwin didn't implement them..but I didn't look. Heck=2C I tried bui= >lding minicom on some non-Linux system(s) recently and it failed miserably.= > I don't know if there is even much Posix portability option for serial por= >t programming. >Sure..I debug over a serial port=2C and lately have machines I can't get vi= >deoout of (Itanium) or even seem to have no video hardware (Sun)=2C so I us= >e a serial consolefor initial install (recent achievement :) still haven't = >installed over network..)=2C but that's still imho a big step away from wri= >ting code that uses the serial port.(And NetBSD doesn't support the local v= >ideo on SGI O2=2C but I'm running OpenBSD there for now.) >=20 >Anyway=2C there are a few ways to look at this issue.More and less trivial. >A trivial view is: Get the filtering to work. Build serial on all platform= >s but Cygwin. Or see if the Posix code will work. > I'd be surprised if the intersection of Cygwin x Modula-3 x serial port p= >rogrammers is not 0=2C but could be. I actually don't think anyone is using= > Cygwin x Modula-3. >Another view is: Get the Win32 serial code to work on Cygwin. The code is= > already there=2C and buildable=2C but building in a system that also build= >s FilePosix.m3 breaks Modula-3 rules=2C you end up with two implementatio= >ns of the File interface. What you want=2C I think=2C is FilePosix.m3 to pr= >ovide the File interface=2C and FileWin32.m3 to provide just its own spec= >ific FileWin32 interface. I think this is a simple transform of the code. >Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sorry.Ma= >ybe I'll get the truncation I deserve. :) > - Jay= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >
>From: Tony
To: jay

>
I would have thought of CygWin as a POSIX platform that just happens t= >o have Windows underneath.  =3BDid we go round this roundabout before?<= >/DIV> >
12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-IND= >ENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPS= >E: separate"> >
e=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B C= >OLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-S= >PACING: normal=3B BORDER-COLLAPSE: separate">pan style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: n= >one=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B = >LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B TEXT-TRAN= >SFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: no= >rmal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">EC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvetica=3B T= >EXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B WHITE-S= >PACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate">class=3DEC_Apple-style-span style=3D"WORD-SPACING: 0px=3B FONT: 12px Helvet= >ica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT: 0px=3B= > WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: separate= >">x Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEXT-INDENT= >: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-COLLAPSE: = >separate">ONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B TEX= >T-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-CO= >LLAPSE: separate">0px=3B FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0= >)=3B TEXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B B= >ORDER-COLLAPSE: separate"> >

V>
>
>
FONT: 12px Helvetica=3B TEXT-TRANSFORM: none=3B COLOR: rgb(0=2C0=2C0)=3B T= >EXT-INDENT: 0px=3B WHITE-SPACE: normal=3B LETTER-SPACING: normal=3B BORDER-= >COLLAPSE: separate"> >
>
It's more complicated than that.
>
>
It isn't just one bit.
>
>
There's a few factors=2C that people care more and less about.
>
>
 =3B GUI library =3B -- Win32 or X
>
>
 =3B thread library =3B -- pthreads or Win32 (or user threads)= >
>
>
 =3B file system path representation -- forward slashes or backwar= >d slashes
 =3B =3B =3B This seems to be what people care mos= >t about -- to see forward slashes.
>
>
 =3B C compiler -- Visual C++ cl or Cygwin/GNU gcc
>
>
 =3B linker -- Visual C++ link or Cygwin/GNU ld
>
>
 =3B C runtime -- msvcr*.dll or cygwin1.dll
>
>
 =3B integrated backend or not
 =3B =3B =3B The in= >tegrated backend is way faster=2C but doesn't yet support 64bit LONGINT.IV> >
>
 =3B which debugger/symbol format =3B-- gdb/stabs or windbg/Vi= >suaStudio/CodeView/pdb
>
>
Many of these factors are independent of each other=2C but not entirel= >y.
>
>
Compiler/linker directly lead to which debugger you can use=2C sinceR> =3B they each use different formats.
>
>
Well=2C you can use either debugger=2C but..without symbols..at least = >for some modules.
>
>
The Cygwin C runtime and Visual C++ linker disagree
on the name of = >__ImageBase=2C so you can't link to the Cygwin C runtime with the
Visual= > C++ linker. This is probably fixable at least in one combination with
a= >n alias (ability to link Cygwin C runtime with Visual C++ linker=3B I don't= >
know if Cygwin/GNU ld understands aliases).
>
>
I have also found that Cygwin produces incorrect
import .libs=2C pe= >rhaps their linker somehow accepts them.
>
> =3B
>
>Cygwin can program the Win32 GUI=2C but there is an issue with passing 8 b= >yte structs by value to __stdcall functions=2C the function name isn't mang= >led correctly and linking fails.
>
>
It is a bit of a combinatorial explosion.
As the system stands=2C y= >ou can control each bit in the config file=2C and
multiple combinations = >might work=2C though I don't run any combinatorial tests.
>
>
Three combinations are "identified" (named=2C spoken of) =3Band ha= >ve had some development testing.
NT386=2C NT386MINGNU (which I think sho= >uld be called I386_MINGWIN)=2C and NT386GNU (which I think should be called= > I386_CYGWIN).
>
>
Furthermore=2C you can imagine I386_INTERIX and I386_UWIN (I forget if= > that's the name).
Interix and UWIN being two other very viable Posix la= >yers on Windows.
Interix has been renamed a few times. It is called "SFU= >" -- services for Unix=2C and I don't know=2C
like SUA -- services for u= >nix applications=2C and I think there's another name.
It is a free downl= >oad from Microsoft for x86 Windows <=3B Vista=2C and is an optional compo= >nent in Vista.
It has its pluses and minuses..not now.
>
>
Anyway=2C in conversation here=2C one of the main things that people D= >O NOT care about
is which thread library is used.
>
>
I believe it was asserted that PM3/NT386GNU did work=2C but that it mi= >ght have used Win32 threads.
>
>And maybe never had any GUI?
>
>
I tried getting pthreads to work=2C but gave up debugging it.
Perha= >ps a fresh go at debugging it would pan out..but I'm really not too interes= >ted..
>
>
I also have no interest in user threads. I believe they are never warr= >anted=2C except
on old systems that don't have kernel threads=2C like th= >e hypothetical DJGPP port.
>
>
I never brought up the issue of serial ports.
I ASSUMED that almost= > nobody uses them=2C and anyone that does use them=2C doesn't
care if th= >ey are implemented using Cygwin or Win32=2C as long as they work.
>
>I ASSUME there is no interop issue=2C like getting an int fd from the seri= >al port
and passing it to select() or close=2C or getting a Win32 HANDLE= > and passing it
to WaitForSingleObject=2C but I could be wrong here=2C c= >ertainly.
>
>
I didn't look into if Cygwin is complete enough to implement them.
= >I consider serial ports to be kind of out on the fringe..that it wouldn't b= >e surprising if Cygwin didn't implement them..but I didn't look. Heck=2C I = >tried building minicom on some non-Linux system(s) recently and it failed m= >iserably. I don't know if there is even much Posix portability option for s= >erial port programming.
>
>
Sure..I debug over a serial port=2C and lately have machines I can't g= >et video
out of (Itanium) or even seem to have no video hardware (Sun)= >=2C so I use a serial console
for initial install (recent achievement :)= > still haven't installed over network..)=2C but that's still imho a big ste= >p away from writing code that uses the serial port.
(And NetBSD doesn't = >support the local video on SGI O2=2C but I'm running OpenBSD there for now.= >)
>
> =3B
>
>
Anyway=2C there are a few ways to look at this issue.
More and less= > trivial.
>
>
A trivial view is:
 =3B Get the filtering to work. Build serial= > on all platforms but Cygwin.
 =3B =3B Or see if the Posix code = >will work.
>
> =3B I'd be surprised if the intersection of Cygwin x Modula-3 x seria= >l port programmers is not 0=2C but could be. I actually don't think anyone = >is using Cygwin x Modula-3.
>
>
Another view is:
 =3B Get the Win32 serial code to work on Cygw= >in.
 =3B The code is already there=2C and buildable=2C but building = >in a system that also builds FilePosix.m3
 =3B =3B breaks Modula= >-3 rules=2C you end up with two implementations of the File interface.
&= >nbsp=3BWhat you want=2C I think=2C is FilePosix.m3 to provide the File inte= >rface=2C and FileWin32.m3 to provide
 =3B =3B just its own speci= >fic FileWin32 interface.
 =3B I think this is a simple transform of = >the code.
>
>
Anyway=2C not a big deal...my tone here...is rushed=2C not rude=2C sor= >ry.
Maybe I'll get the truncation I deserve. :)
>
>
 =3B- Jay
>= > >--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Dec 18 22:21:07 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 18 Dec 2008 13:21:07 -0800 Subject: [M3devel] serial/Cygwin In-Reply-To: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> Message-ID: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Yes, I don't think the world is a terrible place because people like you exist :-) I just think that Cygwin users generally have modest requirements. They're just trying to get something---anything!---to work on Windows, and if they want to do anything that's really Windows-specific I think they/we understand they/we may have to make the effort to learn a bit about Windows and use the native environment. What it adds up to, in my opinion, is that one shouldn't screw things up in the native Windows port (or in the Unix ports) just to satisfy Cygwin users, who really don't care if Windows feature XYZ is available. How many people actually use Modula-3 on Cygwin? I do... and I just use it so I can run programs I developed on Unix on random Windows machines when the owners of those machines have made the bizarre (to me) decision to run Windows... If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y than it is now. For instance, the version I use has Windows file-locking semantics, i.e., default readers-and-writers locking on all open files, which can be quite annoying when porting from Unix. It also uses a different epoch for Time.T. Not sure if the current version has these features... Mika "Randy Coleburn" writes: >--=__Part6F4761B2.0__= >Content-Type: text/plain; charset=US-ASCII >Content-Transfer-Encoding: quoted-printable > >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >about it and Modula-3. >=20 >Mika has a point that if you want to create an environment on Windows that = >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >looking for. >=20 >On the other hand, I am in a camp of folks who don't mind using the = >Microsoft tools and want Modula-3 to work out of the box on Windows = >without having to add any other non-Microsoft stuff (like Cygwin). >=20 >I think it is great for CM3 to be able to support both these camps! >=20 >Thus, you will find me always advocating for keeping the "CM3 on Windows = >variant" where it will work on Windows using just the Microsoft tools. I = >am happy with the use of Windows threads and the native backend and use of = >the MS Visual Studio linker. >=20 >Regards, >Randy Coleburn > >>>> Mika Nystrom 12/17/2008 10:10 PM >>> >Do Cygwin users really care about all that stuff? > >The point of using Modula-3 on Cygwin is to turn a yucky Windows >machine as close as possible into a friendly Unix environment... >not to use a bunch of Microsoft tools and libraries---I would have >thought that if you wanted to use Visual X-Y-Z then you wouldn't >be using Cygwin but would be using the native Windows environment? > >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still >occasionally use on Windows uses Windows threads and has its own >compiler back-end. But it's kind of cool too and since it's not >really distinguishable from user-level threads and gcc I just think >"great" and get on with it. In fact I'm not even sure it uses >the integrated back end now that I think about it. > >I think what I'm saying is that almost no Cygwin users are terribly >interested in Microsoft interfaces, and I would bet most Cygwin >users aren't even aware of the existence of most Microsoft tools. >They (we) want as little to do with Windows as possible.... > > Mika > >Jay writes: >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_ >>Content-Type: text/plain; charset=3D"iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >>From: TonyTo: jay >>I would have thought of CygWin as a POSIX platform that just happens to = >hav=3D >>e Windows underneath. Did we go round this roundabout before? >> >> >> >> >> >>It's more complicated than that. >>It isn't just one bit. >>There's a few factors=3D2C that people care more and less about. >> GUI library -- Win32 or X >> thread library -- pthreads or Win32 (or user threads)=3D20 >> file system path representation -- forward slashes or backward slashes = > =3D >> This seems to be what people care most about -- to see forward slashes. >> C compiler -- Visual C++ cl or Cygwin/GNU gcc >> linker -- Visual C++ link or Cygwin/GNU ld=3D20 >> C runtime -- msvcr*.dll or cygwin1.dll=3D20 >> integrated backend or not The integrated backend is way faster=3D2C = >but=3D >> doesn't yet support 64bit LONGINT. >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView= >/=3D >>pdb >>Many of these factors are independent of each other=3D2C but not = >entirely. >>Compiler/linker directly lead to which debugger you can use=3D2C since = >they =3D >>each use different formats. >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least = >for s=3D >>ome modules. >>The Cygwin C runtime and Visual C++ linker disagreeon the name of = >__ImageBa=3D >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ = >linker. =3D >>This is probably fixable at least in one combination withan alias = >(ability =3D >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if = >Cygwin/GN=3D >>U ld understands aliases). >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps = >the=3D >>ir linker somehow accepts them. >>=3D20 >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing = >8 by=3D >>te structs by value to __stdcall functions=3D2C the function name isn't = >mangl=3D >>ed correctly and linking fails. >>It is a bit of a combinatorial explosion.As the system stands=3D2C you = >can co=3D >>ntrol each bit in the config file=3D2C andmultiple combinations might = >work=3D2C=3D >> though I don't run any combinatorial tests. >>Three combinations are "identified" (named=3D2C spoken of) and have had = >some =3D >>development testing.NT386=3D2C NT386MINGNU (which I think should be = >called I3=3D >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= From jay.krell at cornell.edu Thu Dec 18 23:12:12 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 18 Dec 2008 22:12:12 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y Well, here I can cop-out and say: "Hey, it just uses Cygwin". Other than a possible avenue for the serial code. And other than threads. > Unix. It also uses a different epoch for Time.T. Not sure if I'm pretty sure that's not the case now. It very likely uses 31bit seconds since 1970, and will break around 2038. The hypothetical AMD64_CYGWIN would likely use 63bit seconds since 1970, so better. msvcr*.dll offers 63bits seconds since 1970, you configure it with a #define. (I say 31 or 63, because they are signed numbers probably, but negative numbers aren't considered valid. They really should be unsigned, and with a much earlier epoch...I realize files don't exist before these times, but it's nice to have one time format apply to more than just files, and there's certainly a legitimate need to represent earlier times..) Oh..well, yeah, I did name files foo.lib and not libfoo.a. File naming was another question that came up, along the lines of "What is Cygwin?". People said they didn't care. Similarly, executables named "foo" or "foo.exe"? .dlls named "foo.so" or "foo.dll". Cygwin actually uses foo.dll (or libfoo.dll, er..cygfoo.dll actually) and foo.exe. foo.so works fine, and "foo" works fine, at least on NT. bash will run it. cmd will run it if you add "." to %PATHEXT%. Apparently Win9x doesn't like it, and Cygwin historically supported Win9x (still today, but not for much longer, not with the next major release). So here you can see, that some typical Posix characteristics are not followed by Cygwin. Interix actually does use extensionless executables. You can see the evidence in taskmgr for example. I had it using foo.so (libfoo.so?) and "foo" for a bit, but since people didn't care, and Cygwin didn't do that, I put it back. It was pretty easy to go back and forth, like by fiddling with the "naming conventions" setting. (Which, it turns out, I think, was never really implemented correctly -- there are host and target conventions and I think they were confused.) Wait a sec, this does remind me. The cygfoo.dll convention has the benefit of being able to mix stuff in the same directory or along the same %PATH%. I thought that not useful, since .exes would still clash in name, but maybe? I will put serial back for non-Cygwin, really. And I'll fix SOLsun/gnu. Soon. - Jay> To: rcoleburn at scires.com> To: m3devel at elegosoft.com> Date: Thu, 18 Dec 2008 13:21:07 -0800> From: mika at async.caltech.edu> Subject: Re: [M3devel] serial/Cygwin> > Yes, I don't think the world is a terrible place because people like> you exist :-)> > I just think that Cygwin users generally have modest requirements.> They're just trying to get something---anything!---to work on> Windows, and if they want to do anything that's really Windows-specific> I think they/we understand they/we may have to make the effort to> learn a bit about Windows and use the native environment.> > What it adds up to, in my opinion, is that one shouldn't screw> things up in the native Windows port (or in the Unix ports) just> to satisfy Cygwin users, who really don't care if Windows feature> XYZ is available.> > How many people actually use Modula-3 on Cygwin? I do... and I> just use it so I can run programs I developed on Unix on random> Windows machines when the owners of those machines have made the> bizarre (to me) decision to run Windows...> > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> than it is now. For instance, the version I use has Windows> file-locking semantics, i.e., default readers-and-writers locking> on all open files, which can be quite annoying when porting from> Unix. It also uses a different epoch for Time.T. Not sure if> the current version has these features...> > Mika> > "Randy Coleburn" writes:> >--=__Part6F4761B2.0__=> >Content-Type: text/plain; charset=US-ASCII> >Content-Transfer-Encoding: quoted-printable> >> >I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >about it and Modula-3.> >=20> >Mika has a point that if you want to create an environment on Windows that => >is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >looking for.> >=20> >On the other hand, I am in a camp of folks who don't mind using the => >Microsoft tools and want Modula-3 to work out of the box on Windows => >without having to add any other non-Microsoft stuff (like Cygwin).> >=20> >I think it is great for CM3 to be able to support both these camps!> >=20> >Thus, you will find me always advocating for keeping the "CM3 on Windows => >variant" where it will work on Windows using just the Microsoft tools. I => >am happy with the use of Windows threads and the native backend and use of => >the MS Visual Studio linker.> >=20> >Regards,> >Randy Coleburn> >> >>>> Mika Nystrom 12/17/2008 10:10 PM >>>> >Do Cygwin users really care about all that stuff?> >> >The point of using Modula-3 on Cygwin is to turn a yucky Windows> >machine as close as possible into a friendly Unix environment...> >not to use a bunch of Microsoft tools and libraries---I would have> >thought that if you wanted to use Visual X-Y-Z then you wouldn't> >be using Cygwin but would be using the native Windows environment?> >> >So yes I find it a bit odd that the ancient PM3-Klagenfurt I still> >occasionally use on Windows uses Windows threads and has its own> >compiler back-end. But it's kind of cool too and since it's not> >really distinguishable from user-level threads and gcc I just think> >"great" and get on with it. In fact I'm not even sure it uses> >the integrated back end now that I think about it.> >> >I think what I'm saying is that almost no Cygwin users are terribly> >interested in Microsoft interfaces, and I would bet most Cygwin> >users aren't even aware of the existence of most Microsoft tools.> >They (we) want as little to do with Windows as possible....> >> > Mika> >> >Jay writes:> >>--_c7ada8a5-5684-4472-b0c7-b49c8c279dc2_> >>Content-Type: text/plain; charset=3D"iso-8859-1"> >>Content-Transfer-Encoding: quoted-printable> >>> >>> >>> >>From: TonyTo: jay> >>I would have thought of CygWin as a POSIX platform that just happens to => >hav=3D> >>e Windows underneath. Did we go round this roundabout before?> >>> >>> >>> >>> >>> >>It's more complicated than that.> >>It isn't just one bit.> >>There's a few factors=3D2C that people care more and less about.> >> GUI library -- Win32 or X> >> thread library -- pthreads or Win32 (or user threads)=3D20> >> file system path representation -- forward slashes or backward slashes => > =3D> >> This seems to be what people care most about -- to see forward slashes.> >> C compiler -- Visual C++ cl or Cygwin/GNU gcc> >> linker -- Visual C++ link or Cygwin/GNU ld=3D20> >> C runtime -- msvcr*.dll or cygwin1.dll=3D20> >> integrated backend or not The integrated backend is way faster=3D2C => >but=3D> >> doesn't yet support 64bit LONGINT.> >> which debugger/symbol format -- gdb/stabs or windbg/VisuaStudio/CodeView=> >/=3D> >>pdb> >>Many of these factors are independent of each other=3D2C but not => >entirely.> >>Compiler/linker directly lead to which debugger you can use=3D2C since => >they =3D> >>each use different formats.> >>Well=3D2C you can use either debugger=3D2C but..without symbols..at least => >for s=3D> >>ome modules.> >>The Cygwin C runtime and Visual C++ linker disagreeon the name of => >__ImageBa=3D> >>se=3D2C so you can't link to the Cygwin C runtime with theVisual C++ => >linker. =3D> >>This is probably fixable at least in one combination withan alias => >(ability =3D> >>to link Cygwin C runtime with Visual C++ linker=3D3B I don'tknow if => >Cygwin/GN=3D> >>U ld understands aliases).> >>I have also found that Cygwin produces incorrectimport .libs=3D2C perhaps => >the=3D> >>ir linker somehow accepts them.> >>=3D20> >>Cygwin can program the Win32 GUI=3D2C but there is an issue with passing => >8 by=3D> >>te structs by value to __stdcall functions=3D2C the function name isn't => >mangl=3D> >>ed correctly and linking fails.> >>It is a bit of a combinatorial explosion.As the system stands=3D2C you => >can co=3D> >>ntrol each bit in the config file=3D2C andmultiple combinations might => >work=3D2C=3D> >> though I don't run any combinatorial tests.> >>Three combinations are "identified" (named=3D2C spoken of) and have had => >some =3D> >>development testing.NT386=3D2C NT386MINGNU (which I think should be => >called I3=3D> >>86_MINGWIN)=3D2C and NT386GNU (which I think should be called I386_CYGWIN)= -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Thu Dec 18 23:03:06 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Thu, 18 Dec 2008 16:03:06 -0600 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: <494AC89A.4040601@wichita.edu> I agree that it is highly desirable to have both native windows and Cygwin variants of Modula-3. But I also think that it is unrealistic to try to support a complete or even partial Cartesian product of all the many separate options Jay enumerates. Without debating whether it might be desirable, it's just way too much work when there are other, more pressing things to do with our limited support time. Mix and match is just entirely unrealistic. By the same reasoning, I have given up on the similar ideal of making m3gdb handle a linked-together mix of code from different Modula-3 compilers: PM3, CM3, standalone backend, different versions of gcc-derived back end, etc. I advocate a Cygwin target that is as Unix-like as possible, in as many respects as possible, and a native Windows target that is as Windows-like as possible in as many respects as possible. No more. Mika Nystrom wrote: > Yes, I don't think the world is a terrible place because people like > you exist :-) > > I just think that Cygwin users generally have modest requirements. > They're just trying to get something---anything!---to work on > Windows, and if they want to do anything that's really Windows-specific > I think they/we understand they/we may have to make the effort to > learn a bit about Windows and use the native environment. > > What it adds up to, in my opinion, is that one shouldn't screw > things up in the native Windows port (or in the Unix ports) just > to satisfy Cygwin users, who really don't care if Windows feature > XYZ is available. > > How many people actually use Modula-3 on Cygwin? I do... and I > just use it so I can run programs I developed on Unix on random > Windows machines when the owners of those machines have made the > bizarre (to me) decision to run Windows... > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y > than it is now. For instance, the version I use has Windows > file-locking semantics, i.e., default readers-and-writers locking > on all open files, which can be quite annoying when porting from > Unix. It also uses a different epoch for Time.T. Not sure if > the current version has these features... > > Mika > > "Randy Coleburn" writes: >> --=__Part6F4761B2.0__= >> Content-Type: text/plain; charset=US-ASCII >> Content-Transfer-Encoding: quoted-printable >> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel = >> about it and Modula-3. >> =20 >> Mika has a point that if you want to create an environment on Windows that = >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are = >> looking for. >> =20 >> On the other hand, I am in a camp of folks who don't mind using the = >> Microsoft tools and want Modula-3 to work out of the box on Windows = >> without having to add any other non-Microsoft stuff (like Cygwin). >> =20 >> I think it is great for CM3 to be able to support both these camps! >> =20 >> Thus, you will find me always advocating for keeping the "CM3 on Windows = >> variant" where it will work on Windows using just the Microsoft tools. I = >> am happy with the use of Windows threads and the native backend and use of = >> the MS Visual Studio linker. >> =20 >> Regards, >> Randy Coleburn >> From jay.krell at cornell.edu Fri Dec 19 12:11:54 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 11:11:54 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <494AC89A.4040601@wichita.edu> References: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> <494AC89A.4040601@wichita.edu> Message-ID: Pretty much agreed. The implementation is factored along the lines I describe, but I don't test the cross product. I think there might be one other interesting middle ground -- a target that uses the gcc backend but is otherwise Windows-like..oh, and probably GNU/gcc and GNU/ld, else you can't debug. This is analogous to the "MinGWin" project, which is real and works, it offers gcc, that can I believe build itself, but which generally produces binaries with no extra dependency or "strange" :) behavior. There is a config file for this called NT386MINGNU, though I think it should be called I386_MINGWIN. The system was able to build itself, and getting it to work was a subset of Cygwin -- they shared an important bug. This port can't use Trestle currently due to a minor bug in cm3cg. The benefits of this middle ground are that you get 64bit LONGINT already, and you might get highly optimized code instead of only moderately optimized code. Personally, I almost never ask gcc to optimize. You'll also probably get "AMD64_MINGW" before "AMD64_NT". Neither all that soon, but I know MinGWin/AMD64 is very far along or even done. (Other avenues to avoid updating the backend are C_NT and LLVM_NT, but these are so far just fantasies.) > The implementation is factored along the lines I describe And that is an exaggeration. It is mostly the config file that is factored such. Within the Modula-3 compiler, the only distinction is the "backend mode". The jmpbuf size is chosen I think to be the maximum of the two C runtimes. In the distant distant future, the compiler could also distinguish exception handling codegen strategies. There are several to chose from, like 3 or 4 for x86, 2 for the rest. - setjmp/longjmp -- as it stands now, or interoperable with gcc's similar - "elf/dwarf/whatever" -- range tables, stack walking, gcc supports this these days - NT/x86 style -- builds a linked list through a special per-thread/fiber location conceptually like setjmp/longjmp, but highly optimized. - NT/other -- basically like "elf", but perhaps the relevant data and runtime is different in the details - Jay> Date: Thu, 18 Dec 2008 16:03:06 -0600> From: rodney.bates at wichita.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] serial/Cygwin> > I agree that it is highly desirable to have both native windows and Cygwin> variants of Modula-3. But I also think that it is unrealistic to try to> support a complete or even partial Cartesian product of all the many> separate options Jay enumerates. Without debating whether it might be> desirable, it's just way too much work when there are other, more pressing> things to do with our limited support time.> > Mix and match is just entirely unrealistic. By the same reasoning, I> have given up on the similar ideal of making m3gdb handle a linked-together> mix of code from different Modula-3 compilers: PM3, CM3, standalone> backend, different versions of gcc-derived back end, etc.> > I advocate a Cygwin target that is as Unix-like as possible, in as many> respects as possible, and a native Windows target that is as Windows-like> as possible in as many respects as possible. No more.> > > > Mika Nystrom wrote:> > Yes, I don't think the world is a terrible place because people like> > you exist :-)> > > > I just think that Cygwin users generally have modest requirements.> > They're just trying to get something---anything!---to work on> > Windows, and if they want to do anything that's really Windows-specific> > I think they/we understand they/we may have to make the effort to> > learn a bit about Windows and use the native environment.> > > > What it adds up to, in my opinion, is that one shouldn't screw> > things up in the native Windows port (or in the Unix ports) just> > to satisfy Cygwin users, who really don't care if Windows feature> > XYZ is available.> > > > How many people actually use Modula-3 on Cygwin? I do... and I> > just use it so I can run programs I developed on Unix on random> > Windows machines when the owners of those machines have made the> > bizarre (to me) decision to run Windows...> > > > If anything, I'd like it if the Cygwin Modula-3 were *less* Windows-y> > than it is now. For instance, the version I use has Windows> > file-locking semantics, i.e., default readers-and-writers locking> > on all open files, which can be quite annoying when porting from> > Unix. It also uses a different epoch for Time.T. Not sure if> > the current version has these features...> > > > Mika> > > > "Randy Coleburn" writes:> >> --=__Part6F4761B2.0__=> >> Content-Type: text/plain; charset=US-ASCII> >> Content-Transfer-Encoding: quoted-printable> >>> >> I do use Cygwin on occasion, but I'm not sure how most Cygwin users feel => >> about it and Modula-3.> >> =20> >> Mika has a point that if you want to create an environment on Windows that => >> is devoid of Microsoft tools, the Cygwin Modula-3 may be what you are => >> looking for.> >> =20> >> On the other hand, I am in a camp of folks who don't mind using the => >> Microsoft tools and want Modula-3 to work out of the box on Windows => >> without having to add any other non-Microsoft stuff (like Cygwin).> >> =20> >> I think it is great for CM3 to be able to support both these camps!> >> =20> >> Thus, you will find me always advocating for keeping the "CM3 on Windows => >> variant" where it will work on Windows using just the Microsoft tools. I => >> am happy with the use of Windows threads and the native backend and use of => >> the MS Visual Studio linker.> >> =20> >> Regards,> >> Randy Coleburn> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Dec 19 19:25:52 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 19 Dec 2008 18:25:52 +0000 Subject: [M3devel] serial/Cygwin In-Reply-To: <200812182121.mBILL7cC099435@camembert.async.caltech.edu> References: Your message of "Thu, 18 Dec 2008 15:46:26 EST." <494A6FFD.1E75.00D7.1@scires.com> <200812182121.mBILL7cC099435@camembert.async.caltech.edu> Message-ID: > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay From dabenavidesd at yahoo.es Sat Dec 20 01:09:41 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 20 Dec 2008 00:09:41 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Dear all: I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have. However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not? claim any more they are safe so that "fix" is not feasible. Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway?? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file. Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ),? that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation. There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that? Thanks in advance --- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/Cygwin Para: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25 > And I'll fix SOLsun/gnu. I /think/ it is ok now -- changing my "ino_t" to "m3_ino_t". I'm still checking though. And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Dec 20 08:19:02 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 19 Dec 2008 23:19:02 -0800 Subject: [M3devel] This disgusting TEXT business Message-ID: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Hello Modula-3ers, I have gone on the record before as not being very impressed by the Critical Mass implementation of TEXT. If you bother to read this email to the end, I will show you a simple program that runs 100 times slower when compiled with CM3 than the older PM3. TEXT is a very convenient built-in type, especially I think because it allows literals to be written with a lot less work than literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '\n' } Hence a lot of programs use TEXT liberally. Modula-3 also has a very syntactically convenient TEXT concatenation operator, &. I think it was Guy Steele who observed in one of the Lambda Papers that programmers tend to think syntactically concise notation is associated with efficient implementation. CM3 fails here, and sometimes horribly! I remember about five years ago we were writing a large number of VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" to CM3---I think one of us wanted to run the tools on a Macintosh. Most VLSI CAD tools really do nothing but process strings in various ways. Concatenation, splitting, those sorts of things. Think a large program written in a language without recursion but lots of hierarchical names. Compile unit x containing subunit y containing subunit z containing node n, giving name x.y.z.n . Well to make a long story short, the programmer with the Mac said that CM3 produced horribly slow-running binaries. Something with TEXTs, but we didn't investigate further. We gave up on migrating our group to CM3 and instead use PM3 to this day. The programmer with the Mac has since switched all his work to coding in C++ "in the style of Modula-3." And after seeing what I have seen now I cannot say I will voluntarily use CM3 for anything until we've got a solution. I would personally be happy to rip out everything that says Critical Mass TEXT on it and put back the SRC code. Here's the program I promised: MODULE Main; IMPORT Text, Fmt, IO, Time; CONST Count = 1000*1000; VAR checks := 0; start := Time.Now(); a : TEXT := ""; BEGIN FOR i := 0 TO 100 DO a := a & ":" & Fmt.Int(i) END; LOOP EVAL Text.Equal(a,a); INC(checks); IF checks > Count THEN IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / (Time.Now() - start)) & "\n"); start := Time.Now(); checks := 0 END END END Main. On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: virtual resident Text.Equals machine cycles size size per second compiler per char (approx) --------------------------------------------------------------- 11476K 2396K 13,274 CM3 770 1888K 960K 1,634,000 PM3 6.24 The culprit is TextCat.MyGetWideChars, which calls itself recursively. Here's a representative traceback: Starting program: /big/home/mika/test_text/FreeBSD4/test_text ^C Program received signal SIGINT, Interrupt. 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 111 TextCat.m3: No such file or directory. in TextCat.m3 (m3gdb) where #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:111 #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= [W'8',W':',W'8',W'9',W':',W'9',W'0',W':',W'9',W'1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W '9',W'6',W'5',W':',W'7',W'6',W':',W'7',W'7',W':',W'7',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' :',W'8',W'3',W':',W'8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], start=256) at TextCat.m3:109 #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) at Text.m3:66 #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 #10 0x08048b0d in Main (mode=1) at Main.m3:20 #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) at RTLinker.m3:122 #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at _m3main.mc:4 (m3gdb) Now you may object that perhaps people don't normally make TEXTs by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes they do. There's nothing to warn the user that that is a terribly bad thing to do. Well we know... we think.. it will lead to a lot of memory allocation overhead when you *make* the TEXT. But that it will infect the generated TEXT forevermore with this long chain of pointers is, to say the least, surprising. The reason I noticed this now is that I wrote a program that "watches" a directory of files (using FS.Iterate). It even uses a hash table to store the TEXTs. *Still* it is an incredible CPU hog (on Linux wth CM3). I think I just did path := dir & "/" & filename and that's enough to make it run 10-30 times slower than a PM3-compiled binary from the same source code (for suitably pathological choices of dir and filename). Does anyone have a better idea than just throwing away the CM3 TEXT implementation? Mika From hosking at cs.purdue.edu Sat Dec 20 09:26:40 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 20 Dec 2008 19:26:40 +1100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Hmm, are we just victims of poor implementation? Does anyone have the time to improve things? It would be possible to rip out CM3 TEXT and replace with PM3, but we'd lose WIDECHAR and WideText.T with that too. Not sure who that impacts. On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. > > If you bother to read this email to the end, I will show you a > simple program that runs 100 times slower when compiled with > CM3 than the older PM3. > > TEXT is a very convenient built-in type, especially I think because > it allows literals to be written with a lot less work than > > literal := ARRAY OF CHAR { 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', > 'r', 'l', 'd', '!', '\n' } > > Hence a lot of programs use TEXT liberally. > > Modula-3 also has a very syntactically convenient TEXT concatenation > operator, &. > > I think it was Guy Steele who observed in one of the Lambda Papers > that > programmers tend to think syntactically concise notation is associated > with efficient implementation. > > CM3 fails here, and sometimes horribly! > > I remember about five years ago we were writing a large number of > VLSI CAD tools in Modula-3 at Caltech. We were going to "upgrade" > to CM3---I think one of us wanted to run the tools on a Macintosh. > Most VLSI CAD tools really do nothing but process strings in various > ways. Concatenation, splitting, those sorts of things. Think a > large program written in a language without recursion but lots of > hierarchical names. Compile unit x containing subunit y containing > subunit z containing node n, giving name x.y.z.n . > > Well to make a long story short, the programmer with the Mac said > that CM3 produced horribly slow-running binaries. Something with > TEXTs, but we didn't investigate further. We gave up on migrating > our group to CM3 and instead use PM3 to this day. The programmer > with the Mac has since switched all his work to coding in C++ "in > the style of Modula-3." > > And after seeing what I have seen now I cannot say I will voluntarily > use CM3 for anything until we've got a solution. I would personally > be happy to rip out everything that says Critical Mass TEXT on it and > put back the SRC code. > > Here's the program I promised: > > MODULE Main; > IMPORT Text, Fmt, IO, Time; > > CONST > Count = 1000*1000; > > VAR > checks := 0; > start := Time.Now(); > a : TEXT := ""; > BEGIN > > FOR i := 0 TO 100 DO > a := a & ":" & Fmt.Int(i) > END; > > LOOP > EVAL Text.Equal(a,a); > INC(checks); > IF checks > Count THEN > IO.Put("rate = " & Fmt.LongReal(FLOAT(Count,LONGREAL) / > (Time.Now() - start)) & "\n"); > start := Time.Now(); > checks := 0 > END > END > > END Main. > > On the same machine, running FreeBSD 5.5-RELEASE with a Core2 Duo > E8400 @ 3.00 GHz and 2 GB of RAM, I get the following performance: > > virtual resident Text.Equals machine cycles > size size per second compiler per char (approx) > --------------------------------------------------------------- > 11476K 2396K 13,274 CM3 770 > 1888K 960K 1,634,000 PM3 6.24 > > The culprit is TextCat.MyGetWideChars, which calls itself recursively. > Here's a representative traceback: > > Starting program: /big/home/mika/test_text/FreeBSD4/test_text > ^C > Program received signal SIGINT, Interrupt. > 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > 111 TextCat.m3: No such file or directory. > in TextCat.m3 > (m3gdb) where > #0 0x681f51f1 in MyGetWideChars (t=16_68b188a8, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:111 > #1 0x681f51b3 in MyGetWideChars (t=16_68b188c4, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #2 0x681f51b3 in MyGetWideChars (t=16_68b188e0, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #3 0x681f51b3 in MyGetWideChars (t=16_68b188fc, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #4 0x681f51b3 in MyGetWideChars (t=16_68b18918, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #5 0x681f51b3 in MyGetWideChars (t=16_68b18934, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #6 0x681f51b3 in MyGetWideChars (t=16_68b18950, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #7 0x681f51b3 in MyGetWideChars (t=16_68b1896c, a= > > [W > '8 > ',W > ':',W > '8 > ',W > '9 > ',W > ':',W > '9 > ',W > '0 > ',W > ':',W > '9 > ',W > '1',W':',W'9',W'2',W':',W'9',W'3',W':',W'9',W'4',W':',W'9',W'5',W':',W > '9 > ',W > '6 > ',W > '5 > ',W > ':',W > '7 > ',W > '6 > ',W > ':',W > '7 > ',W > '7 > ',W > ':',W > '7 > ',W'8',W':',W'7',W'9',W':',W'8',W'0',W':',W'8',W'1',W':',W'8',W'2',W' > :',W > '8 > ',W > '3 > ',W > ':',W > '8',W'4',W':',W'8',W'5',W':',W'8',W'6',W':',W'8',W'7',W':',W'8'], > start=256) at TextCat.m3:109 > #8 0x681f0d23 in EqualBuf (t=16_68b1896c, u=16_68b1896c, len=294) > at Text.m3:66 > #9 0x681f0b05 in Equal (t=16_68b1896c, u=16_68b1896c) at Text.m3:39 > #10 0x08048b0d in Main (mode=1) at Main.m3:20 > #11 0x681d24e2 in RunMainBody (m=16_08049d60) at RTLinker.m3:399 > #12 0x681d1a0d in AddUnitI (m=16_08049d60) at RTLinker.m3:113 > #13 0x681d1a94 in AddUnit (b={"Main_M3", Declared at: Main.m3:13}) > at RTLinker.m3:122 > #14 0x08048a41 in main (argc=1, argv=0xbfbfe2a8, envp=0xbfbfe2b0) at > _m3main.mc:4 > (m3gdb) > > Now you may object that perhaps people don't normally make TEXTs > by concatenating 100 small sub-TEXTs. Maybe not.. but sometimes > they do. There's nothing to warn the user that that is a terribly > bad thing to do. Well we know... we think.. it will lead to a lot > of memory allocation overhead when you *make* the TEXT. But that > it will infect the generated TEXT forevermore with this long chain > of pointers is, to say the least, surprising. > > The reason I noticed this now is that I wrote a program that "watches" > a directory of files (using FS.Iterate). It even uses a hash table > to store the TEXTs. *Still* it is an incredible CPU hog (on Linux > wth CM3). I think I just did > > path := dir & "/" & filename > > and that's enough to make it run 10-30 times slower than a PM3- > compiled > binary from the same source code (for suitably pathological choices > of dir and filename). > > Does anyone have a better idea than just throwing away the CM3 TEXT > implementation? > > Mika From dragisha at m3w.org Sat Dec 20 09:00:54 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 09:00:54 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812200719.mBK7J242062696@camembert.async.caltech.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> Message-ID: <1229760054.18256.134.camel@faramir.m3w.org> On Fri, 2008-12-19 at 23:19 -0800, Mika Nystrom wrote: > Hello Modula-3ers, > > I have gone on the record before as not being very impressed by the > Critical Mass implementation of TEXT. If I ever saw an understatement here - then this is one :-) To make long story even shorter - what CM3 tried to do with their WIDECHAR bussiness and extra-fast concatenations was obsoleted very fast after (and some of it before) they put all that hell of code in their codebase. Whatever they thought to do - UTF-8 is doing with simple byte arrays, and string operations like array move are fastest non-trivial operations on CPUs since Z80. I hope we will see some end of it in times future. TIA. -- Dragi?a Duri? From dragisha at m3w.org Sat Dec 20 10:10:45 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 10:10:45 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> Message-ID: <1229764245.18256.148.camel@faramir.m3w.org> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either slow, or produces results that make subsequent operations slow - is where problem is with this implementation. IMO, best solution would be to replace internal representation with UTF-8. For whom may be concerned with it - make some external widechar conversion routines available. That way - concat would be as fast as it can be made and other operations would be realistic - it is how almost everybody does their Unicode strings, after all. Almost everybody - excluding mobile industry and Microsoft :-), AFAIK. Compiler changes would make transparent conversion of literals. Everything else is already transparent with regard to internal representation. I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant to accept them as he did not believe C base routines were widespread enough. GNU world has no such reluctance. Everything is UTF8. If nobody can fill this job in next few weeks, I will probably have some time this winter. No promise, of course, but list will know :-). dd On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: > Hmm, are we just victims of poor implementation? Does anyone have the > time to improve things? It would be possible to rip out CM3 TEXT and > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > too. Not sure who that impacts. > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote: > > > Hello Modula-3ers, > > > > I have gone on the record before as not being very impressed by the > > Critical Mass implementation of TEXT. > > -- Dragi?a Duri? From jay.krell at cornell.edu Sat Dec 20 17:19:49 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 20 Dec 2008 16:19:49 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: My opinions.. UTF8 is a hack. It lets people do approximately nothing and claim they have done a lot. UTF8 is popular, but I'm still very skeptical of it. The best representation for Unicode is fixed size 16 bit characters, as Windows uses "everywhere". All the functions taking 8 bit characters are just thin wrappers over functions dealing with 16 bit characters. I believe NTFS stores 16 bit strings natively on disk. And so does FAT -- anything within a limited character set gets a regular old 8 bit 8.3 entry on disk. Anything "long", or with "other characters", or other forms like anything with two dots, I believe is stored as full 16 bit characters. In fact, I believe if you write an installable file system on Win9x, you deal with all 16 bit characters. What Windows does with 8 bit strings if of course much worse than UTF8. 8 bit characters on Windows are often considered "encoded in the current code page", with usually no record of the code page. I think probably the best thing to do with 8 bit characters is actually require them to be 7 bits. Or even 5 or 6 six bits, and conversion between them is just zero extension or 8 bit truncation. UTF8 is progress because it is, like, one universal code page. UTF8 is slightly muddied because Java defines it differently. Specifically I think there is an issue as to how to represent an embedded nul. There is also UTF7. I guess that's only useful in transports that aren't 8 bit clean. Is UTF a "transport" mechanism only, for an encoding to leave data in as much as possible? That is gray of course. "Entering" and "exiting" from "transport" is expensive when done in bulk, and avoided for perf. The best representation for a "string" is a length and an array of 16 bit characters, probably via a separate pointer, not rigth after the length. Arguably you need a 32 bit character since Unicde is, I believe, actually a 20 bit encoding. However here I'm willing to see things the other way. (ie: maybe hypocritical) 8 bit characters often suffice and it is probably worth being somewhat "dynamic". 8 bit bytes are also common, but should not be confused with characters and strings. Whatever functions any UTF8 code depends on, can be trivially implemented one's self, in portable C or portable Modula-3. How about.. I realized the Modula-3 code is "dynamic" and can deal with 8 or 16 bit characters. I didn't realize it concated strings via keeping around the pointers. This is actually a viable faster approach sometimes. They are called "ropes" in some contexts -- such as the SGI STL. I have to read the code, but..two obvious suggestions I am sympathetic to: 1) Leave things mostly alone, but add more functions people can call. Like String.Flatten, String.Cleanup, String.Seal. That would walk all the pointers and copy the data into one flat string. "Seal" is an exaggeration, since you could subsequently change the string. Problem with this approach of course is that any code that is slowed down, remains slowed down. You still have to do a little work to get back the perf. However, any code sped up by the current code remains sped up. 2) Always "flatten" upon concat. Remain dynamic in the representation, based on what characters are seen. Once a character above 127 is seen, the string is made wide. Any operation on a wide string that both returns a subset of it, AND has to anyway visit every character, can look for the opportunity to shorten it, IF a separate copy is made. Operations such as taking a prefix or suffix -- which can be made by just bumping the pointer or length, do NOT need to visit the characters and would not opportunistically narrow. Can taking a suffix like that work, with the garbage collector? Furthermore, the expression: foo & "a" & bar & "b" should be transformed, by the compiler, into one function call. If not currently, then "to do". This alone, with nothing else, might help a lot? Among the base-most string operations is not or shall not be "concat two strings", but rather "concat n strings". Hm. I think there's a third good option. Basically again looking to Windows. Windows has no one true string type..while internally Unicode is universal, all string functions are doubled up. For any given function foo, say for example CreateFile, there are actually two functions, CreateFileA and CreateFileW. To make a string literal in C or C++, you either say "foo" for an 8 bit string, or L"foo" for 16 bits. String length -- wcslen and strlen -- these names come from standard C and there are a bunch of them. pro: no existing code changes meaning con: no existing code changes meaning Anyone who needs unicode, needs to change their code, somehow. There is a static indirection mechanism. #ifdef UNICODE #define TCHAR WCHAR #define TEXT(a) L pasted to a #define CreateFile CreateFileW etc. #else #define TCHAR char #define a #define CreateFile CreateFileA etc. #endif You can write code to be portable either way. This was a dubious idea, because most code only compiles one way or the other. But it does let you migrate slowly -- keep 8 bit strings but write your code with the future in mind, and then later throw the switch. Personally I just hardcode everything WCHAR, CreateFileW, no illusions. Anyway, I bring this up to try to come up with a proposal along the lines of: - Take everything back to the way it was. - Offer some new way to support Unicode. but I'm not sure what the second part is. Is TEXT abstract enough that it can do double duty? What sort of interop is needed between code using narrow text and code using wide text? Must it be "automatic"? While I say UTF8 is a hack, I like something almost the same as Dragi?a says. Keep everything as 7 or 8 bit characters. Provide conversion routines. Make no assumptions internally that the characters are UTF8. Any internal string as array of char walking would just use individual bytes. Or a "decoding callback". ? Reject any text literal that isn't 7 or 8 bit clean. ? I don't know, I guess I'm very on the fence. Unicode is important and all, interop seems maybe important, it is hard to get anyone to change existing code..what CM3 did at least was seamless..in terms of no code change, well, er..perf change, and they barred direct access to the characters, which might still be good. Maybe you can ask for a direct pointer, of a specific type, and if you ask for the wrong type, you get NULL? ?? Now, this email contains no code to back it up. :) - Jay> From: dragisha at m3w.org> To: m3devel at elegosoft.com> Date: Sat, 20 Dec 2008 10:10:45 +0100> Subject: Re: [M3devel] This disgusting TEXT business> > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either> slow, or produces results that make subsequent operations slow - is> where problem is with this implementation.> > IMO, best solution would be to replace internal representation with> UTF-8. For whom may be concerned with it - make some external widechar> conversion routines available.> > That way - concat would be as fast as it can be made and other> operations would be realistic - it is how almost everybody does their> Unicode strings, after all. Almost everybody - excluding mobile industry> and Microsoft :-), AFAIK.> > Compiler changes would make transparent conversion of literals.> Everything else is already transparent with regard to internal> representation.> > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant> to accept them as he did not believe C base routines were widespread> enough. GNU world has no such reluctance. Everything is UTF8.> > If nobody can fill this job in next few weeks, I will probably have some> time this winter. No promise, of course, but list will know :-).> > dd> > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote:> > Hmm, are we just victims of poor implementation? Does anyone have the > > time to improve things? It would be possible to rip out CM3 TEXT and > > replace with PM3, but we'd lose WIDECHAR and WideText.T with that > > too. Not sure who that impacts.> > > > On 20 Dec 2008, at 18:19, Mika Nystrom wrote:> > > > > Hello Modula-3ers,> > >> > > I have gone on the record before as not being very impressed by the> > > Critical Mass implementation of TEXT.> > >> > -- > Dragi?a Duri? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Dec 20 19:20:12 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 20 Dec 2008 19:20:12 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <1229797212.18256.157.camel@faramir.m3w.org> On Sat, 2008-12-20 at 16:19 +0000, Jay wrote: > My opinions.. > > UTF8 is a hack. > It lets people do approximately nothing and claim they have done a > lot. > UTF8 is popular, but I'm still very skeptical of it. I am very skeptical to skepticism without arguments. Extremelly skeptical when argument is "Windows does it...". > The best representation for Unicode is fixed size 16 bit characters, > as Windows > uses "everywhere". All the functions taking 8 bit characters are > just thin > wrappers over functions dealing with 16 bit characters. And both these facts/argument/whatever sounds scary, at least for me, me being a person of Modula-2/Modula-3 upbringing :-). UTF-8 has done everything it promised. UTF-8 is everywhere, except in Windows and mobile industry. This I/O conversion routines will do when cooperating with these worlds. As you pointed out - 16bit is not enoough for Unicode. UTF-8 covers all Unicode characters. If only this one is argument for Modula-3 Unicode representation in 2009 - it is enough for me. -- Dragi?a Duri? From roland.illig at gmx.de Sat Dec 20 19:38:01 2008 From: roland.illig at gmx.de (Roland Illig) Date: Sat, 20 Dec 2008 19:38:01 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <494D3B89.3020802@gmx.de> Dragi?a Duri? schrieb: > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. I object. UTF-8 is a good external representation format, but in memory, it is far more efficient to let all characters of a string have the same size. Java, for example, has defined a character to be "what fits into 16 bits". That's good enough in many situations. One of the situations it fails is when you want to represent Domino Tiles or Mahjong Tiles as characters. To give you some arguments against UTF-8 as in-memory representation: * How fast can you calculate the length of a string? (With length I mean the number of characters, not the memory size needed for storing them.) * Where does substring(5) start? Where does substring(10000) start, and how long does it take to find out? An argument for the UTF-8 representation is that it saves memory -- one byte for every ASCII character. Is this argument still as important as it has been 40 years ago? Roland From wagner at elegosoft.com Sat Dec 20 20:27:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 20 Dec 2008 20:27:55 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Hi everybody, I haven't followed all the discussions recently, but think it may help to comment on a few things here. IIRC the main reason for the current TEXT implementation was to become compatible with Java. Java strings and M3 strings should look the same for the embedded JVM written in M3. AFAIK this JVM has never been made open source nor is there any intention to do it. Still it may be a good idea to adhere to the TEXT representation if we ever intend to combine Java and M3 in one runtime. I don't think the implementation of the combined 8/16 bit TEXTs is very mature. I think that some improvements could be made. Has anybody looked into the details of the implementation? As it is already pointed out on the old CM3 notes on the web pages, some things are slower while others are now faster. I agree that the implementation is rather suboptimal for some common use cases. We'd need a set of use cases and tests agreed upon to really compare implementations though. Just one test for cat is not enough IMO. This should be the first step for an improvement. I'd also not object to a separation of the 8 and 16 bit implementations, but the type WIDECHAR etc. should remain at least for compatibility. We could perhaps even add some automatic conversion then between TEXT and WIDETEXT types (though this would not be in the M3 spirit :-) If we follow this way, the standard TEXTs could be replaced by the old PM3 implementation again. I'd also like to point out that much of the speed of using the old implementation comes by using the internal TextF interface, which exposes the internal TEXT representation as an array of CHAR. I'm not sure if this is really a good idea. Replacing direct array access by Text.GetChar was one of the main adaptions needed when converting all the standard packages to the CM3 compiler for the first release. I'd rather object to exposing this interface again. We could even define the standard TEXT representation as UTF8 in CM3 instead of the standard ISO Latin1 code set, which seems a bit euro- centric ;-) I'd support this idea, once we have proper UTF8 support in CM3 (which we currently haven't if I am not mistaken, see below). Quoting Dragi?a Duri? : > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. > Everything else is already transparent with regard to internal > representation. I am not sure that it is as easy as that. I'd like to see a proper update of the specification before we implement such a change. > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. No, the problem was that what I got couldn't even be compiled on FreeBSD and Solaris. We should not rely on some external GNU libraries, but we need to implement the UTF8 functions in CM3. Has anybody done that yet? > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). I'd be in favour of any improvements of the TEXT implementation and UTF8 support in CM3! > dd > > On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >> Hmm, are we just victims of poor implementation? Does anyone have the >> time to improve things? It would be possible to rip out CM3 TEXT and >> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >> too. Not sure who that impacts. I'd be careful here, too. I think we need to understand the problems and the impacts of changes better before we rip out much code. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From martinbishop at bellsouth.net Sat Dec 20 22:45:36 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sat, 20 Dec 2008 15:45:36 -0600 Subject: [M3devel] CM3 Source "pure"? Message-ID: <494D6780.6060701@bellsouth.net> I was looking through the CM3 sources out of boredom and noticed that there seems to be no C code, not even for runtime stuff? Are the CM3 sources "pure" in that they don't use any C code directly? (I did see some uses of M3toC and such). From hosking at cs.purdue.edu Sat Dec 20 23:05:18 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 21 Dec 2008 09:05:18 +1100 Subject: [M3devel] CM3 Source "pure"? In-Reply-To: <494D6780.6060701@bellsouth.net> References: <494D6780.6060701@bellsouth.net> Message-ID: <4D272586-0C33-4B7B-B8E6-7BD6A34C1B23@cs.purdue.edu> On 21 Dec 2008, at 08:45, Martin Bishop wrote: > I was looking through the CM3 sources out of boredom and noticed that > there seems to be no C code, not even for runtime stuff? Precisely, Modula-3 is a systems programming language just as capable as C. To that end, a goal of any language implementation is to "eat your own dog food" and implement as much of the language in itself as possible. > Are the CM3 sources "pure" in that they don't use any C code directly? > (I did see some uses of M3toC and such). There are some C sources that adapt to C library and system functions, but generally use of C is discouraged in the Modula-3 implementation. From darko at darko.org Sun Dec 21 00:08:57 2008 From: darko at darko.org (Darko) Date: Sun, 21 Dec 2008 08:08:57 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: The right way to do this, IMNSHO is to not assume any particular representation of TEXT values and create an implementation interface that allows implementations of multiple text representations, much like Rd and Wr don't make many assumptions about how data is actually stored or retrieved. This allows: - backward compatibility with the existing scheme - people's favourite representations (I personally don't think UTF-8 or UTF-16 are much chop as a single solution) - compatibility with external representation requirements (not having to convert each time you call an external API) - optimisation for particular application implementations The complication is with allowing representations to change at runtime and (automatic) conversion between representations, which I don't think adds too much complexity for the possible benefit. Also we need a proper UNICODE type that can contain only valid Unicode characters since WIDECHAR can't represent them all in one value. On 21/12/2008, at 4:27 AM, Olaf Wagner wrote: > Hi everybody, > > I haven't followed all the discussions recently, but think it may help > to comment on a few things here. > > IIRC the main reason for the current TEXT implementation was to become > compatible with Java. Java strings and M3 strings should look the same > for the embedded JVM written in M3. AFAIK this JVM has never been made > open source nor is there any intention to do it. > Still it may be a good idea to adhere to the TEXT representation if > we ever intend to combine Java and M3 in one runtime. > > I don't think the implementation of the combined 8/16 bit TEXTs is > very mature. I think that some improvements could be made. Has anybody > looked into the details of the implementation? As it is already > pointed > out on the old CM3 notes on the web pages, some things are slower > while > others are now faster. I agree that the implementation is rather > suboptimal > for some common use cases. We'd need a set of use cases and tests > agreed > upon to really compare implementations though. Just one test for cat > is > not enough IMO. This should be the first step for an improvement. > > I'd also not object to a separation of the 8 and 16 bit > implementations, > but the type WIDECHAR etc. should remain at least for > compatibility. We could perhaps even add some automatic conversion > then between TEXT and WIDETEXT types (though this would not be in the > M3 spirit :-) If we follow this way, the standard TEXTs could be > replaced by the old PM3 implementation again. > > I'd also like to point out that much of the speed of using the old > implementation comes by using the internal TextF interface, which > exposes the internal TEXT representation as an array of CHAR. I'm > not sure if this is really a good idea. Replacing direct array access > by Text.GetChar was one of the main adaptions needed when converting > all the standard packages to the CM3 compiler for the first release. > I'd rather object to exposing this interface again. > > We could even define the standard TEXT representation as UTF8 in CM3 > instead of the standard ISO Latin1 code set, which seems a bit euro- > centric ;-) I'd support this idea, once we have proper UTF8 support > in CM3 (which we currently haven't if I am not mistaken, see below). > > Quoting Dragi?a Duri? : > >> Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is >> either >> slow, or produces results that make subsequent operations slow - is >> where problem is with this implementation. >> >> IMO, best solution would be to replace internal representation with >> UTF-8. For whom may be concerned with it - make some external >> widechar >> conversion routines available. >> >> That way - concat would be as fast as it can be made and other >> operations would be realistic - it is how almost everybody does their >> Unicode strings, after all. Almost everybody - excluding mobile >> industry >> and Microsoft :-), AFAIK. >> >> Compiler changes would make transparent conversion of literals. >> Everything else is already transparent with regard to internal >> representation. > > I am not sure that it is as easy as that. I'd like to see a proper > update of the specification before we implement such a change. > >> I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was >> reluctant >> to accept them as he did not believe C base routines were widespread >> enough. GNU world has no such reluctance. Everything is UTF8. > > No, the problem was that what I got couldn't even be compiled on > FreeBSD and Solaris. We should not rely on some external GNU > libraries, > but we need to implement the UTF8 functions in CM3. Has anybody done > that yet? > >> If nobody can fill this job in next few weeks, I will probably have >> some >> time this winter. No promise, of course, but list will know :-). > > I'd be in favour of any improvements of the TEXT implementation and > UTF8 support in CM3! > >> dd >> >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have >>> the >>> time to improve things? It would be possible to rip out CM3 TEXT >>> and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > > I'd be careful here, too. I think we need to understand the problems > and the impacts of changes better before we rip out much code. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From mika at async.caltech.edu Sun Dec 21 01:02:11 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 20 Dec 2008 16:02:11 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Sat, 20 Dec 2008 20:27:55 +0100." <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Olaf Wagner writes: ... >> On Sat, 2008-12-20 at 19:26 +1100, Tony Hosking wrote: >>> Hmm, are we just victims of poor implementation? Does anyone have the >>> time to improve things? It would be possible to rip out CM3 TEXT and >>> replace with PM3, but we'd lose WIDECHAR and WideText.T with that >>> too. Not sure who that impacts. > >I'd be careful here, too. I think we need to understand the problems >and the impacts of changes better before we rip out much code. > I do agree. My comments about simply putting back the PM3 code weren't meant to be taken seriously. I myself have always disliked this "internationalization" business. I remember my father used to have to prefix prices in Telexes to England with an o-with-umlaut (the o-with-umlaut on his Swedish Siemens printed as a lira on a British machine). And there's always been something vaguely Socialistic about it: I don't mean the fact that I had to use A-umlaut and Scandinavian-A-with-circle for square brackets on the C-64, but that on the very first computer I ever used, the Swedish Luxor ABC-80 (Zilog Z80 based), the Swedish standard of the time called for the dollar sign to be replaced by a small "sun symbol" pronounced "sol" in Swedish. BASIC strings were A-sol and B-sol, etc. I do not believe the "sol" is any more useful than the "dollar" in Sweden (probably less, especially in the winter); I have a feeling that Swedish standards bodies at the time were simply trying to show their sympathies for the money-less regime of the Khmer Rouge. But that being said, I realize there are people out there who might want non-ASCII in their strings. I think if the CM3 implementation does that well it might be worth fixing... not sure that UTF encoding is any simpler, overall...? A peek at Text.Equal sheds some light on the problem: PROCEDURE Equal (t, u: T): BOOLEAN = VAR info_t, info_u: Info; BEGIN t.get_info (info_t); u.get_info (info_u); IF (info_t.length # info_u.length) THEN RETURN FALSE; END; IF (info_t.length = 0) THEN RETURN TRUE; END; IF (info_t.start = NIL) OR (info_u.start = NIL) OR (info_t.wide # info_u.wide) THEN RETURN EqualBuf (t, u, info_t.length); ELSIF NOT info_t.wide THEN RETURN String8.Equal (info_t.start, info_u.start, info_t.length); ELSE RETURN String16.Equal (info_t.start, info_u.start, info_t.length); END; END Equal; Under CM3, Text.Cat/RTHooks.Concat (&) returns a special subtype of string, viz., TextCat.T, which has the following for get_info: PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; The NIL for info.start is what pushes the code into EqualBuf, which compares the strings character by character, but asks for each character via the get_wide_chars method, which calls itself recursively for each concatenated component. info.start = NIL is, in short, a flag that means "non-standard TEXT, use heavily object-oriented character-by-character routines". It seems to me this implementation could be sped up a great deal by doing some simple micro-optimizations. Let me ask a very simple question... is it possible to sub-type TEXT in CM3? (Is such an interface exported?) If not, it would be quite easy to de-object-orient this code and get most of the PM3 performance back, I think (we'd have to flatten the strings a bit too). If the code can be subtyped by clients outside of the text package it would be necessary to sort out what types of overrides should be supported, and what should happen if someone, for instance, subtypes a TextCat.T to have a different method for getting characters... I will try to get some time to look at this myself over the holidays. Note to self and others: m3gdb is not completely useful here. If you look at the structure of a TextCat.T, you see that fields a and b are TEXTs, but m3gdb doesn't really let you look inside them, since, well, they are TEXTs and printed as such. (You don't get to see the whole gory mess of pointers.) Is there a way to turn off this (normally very helpful) m3gdb feature? Mika From hendrik at topoi.pooq.com Sun Dec 21 21:40:03 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Sun, 21 Dec 2008 15:40:03 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229764245.18256.148.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> Message-ID: <20081221204002.GA26711@topoi.pooq.com> On Sat, Dec 20, 2008 at 10:10:45AM +0100, Dragi?a Duri? wrote: > Whole bussiness of mixed TEXT's - concat of TEXT and WIDETEXT is either > slow, or produces results that make subsequent operations slow - is > where problem is with this implementation. > > IMO, best solution would be to replace internal representation with > UTF-8. For whom may be concerned with it - make some external widechar > conversion routines available. UTF-8 is indeed designed so that a lot of operations can be oerformed directly on the bytes of UTF-8. For example, if you sort UTF-8 strings based on the 8-bit unsigned values of their bytes, you get the same order as if you sorted them based on the integer values of their Unicode characters. As for delimiting strings based on character in them, such as single-byte spaces, you can just process it as a byte stream. If you want to find a character that's, say, three UTF8 bytes, you can do ordinary byte-string search for a three-byte string -- UTF-8 has self-delimiting characters. > > That way - concat would be as fast as it can be made and other > operations would be realistic - it is how almost everybody does their > Unicode strings, after all. Almost everybody - excluding mobile industry > and Microsoft :-), AFAIK. > > Compiler changes would make transparent conversion of literals. I worked on a C compiler long ago, and read the standard in great detail to determine that quoted strings contained characters, not bytes. As a result, the compiler we wrked on had a bug when it came to handling Korean characters when working in a Korean environ -- one of the Korean two-byte characters happened to contain a null byte, and the compiler treated it internally as end-of-string. Of course, we fixed, that, using locale-dependent character-parsing routines. Now Unicode doesn't even have that problem -- the only character containing a null byte is the null character, and it's a single null byte. (Java messes this up by using a different encoding for the null character, so that they can treat their compactly-encoded strings as being null-terminated. But they don't claim to used UTF-8 either, though they do almost use it. The encoding of the null character they use is such that almost any UTF-8 DEcoder will decode it as being a NULL character, unless it is specifically designed to check for it as an error condition. Just look ar the high-oprder bits of the bytes and do your shifts and you'll be OK.) I routinely use UTF-8 as the normal character code in all the software I write, and I almost never have an occasion when library-provided encoding and decoding functions are of much use. If I care much about the syntax, I'm using a parser anyway, and a parser can just as easlily parse bytes with the high bit set as bytes without. -- hendrik > Everything else is already transparent with regard to internal > representation. > > I've sent some UTF-8 routines ages ago to Olaf. IIRC, he was reluctant > to accept them as he did not believe C base routines were widespread > enough. GNU world has no such reluctance. Everything is UTF8. > > If nobody can fill this job in next few weeks, I will probably have some > time this winter. No promise, of course, but list will know :-). What's the job in question? Code that handles UTF-8? New data structures just for UTF-8 strings? Or code that just processes ordinary strings and does special processing when indicated on bytes with the high bit set? -- hendrik From dragisha at m3w.org Sun Dec 21 11:33:18 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 21 Dec 2008 11:33:18 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <494D3B89.3020802@gmx.de> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> Message-ID: <1229855598.18256.168.camel@faramir.m3w.org> Most important argument for UTF-8 - IMO - would be it's completeness. 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit storage magic instead? Accepting 2 byte would be same mistake once over. Once Latin1 was whole world. Is it 2byte now? As for "length" argument - it is cacheable data. As our TEXT is object type, we can do such things. As for "substr", it is sound counter argument. Internal optimizations are possible, of course. For lengthy strings, we can implement it, and efficiently. Once internal representation of TEXT was possible to pass to C without conversion. With UTF8 - it's possible again. Java is not world around us - it's C. And again - it's not it's storage efficiency that coounts - it's completeness. On Sat, 2008-12-20 at 19:38 +0100, Roland Illig wrote: > Dragi?a Duri? schrieb: > > IMO, best solution would be to replace internal representation with > > UTF-8. For whom may be concerned with it - make some external widechar > > conversion routines available. > > > > That way - concat would be as fast as it can be made and other > > operations would be realistic - it is how almost everybody does their > > Unicode strings, after all. Almost everybody - excluding mobile industry > > and Microsoft :-), AFAIK. > > I object. UTF-8 is a good external representation format, but in memory, > it is far more efficient to let all characters of a string have the same > size. Java, for example, has defined a character to be "what fits into > 16 bits". That's good enough in many situations. One of the situations > it fails is when you want to represent Domino Tiles or Mahjong Tiles as > characters. > > To give you some arguments against UTF-8 as in-memory representation: > > * How fast can you calculate the length of a string? (With length I mean > the number of characters, not the memory size needed for storing them.) > > * Where does substring(5) start? Where does substring(10000) start, and > how long does it take to find out? > > An argument for the UTF-8 representation is that it saves memory -- one > byte for every ASCII character. Is this argument still as important as > it has been 40 years ago? > > Roland -- Dragi?a Duri? From stsp at elego.de Sun Dec 21 16:40:15 2008 From: stsp at elego.de (Stefan Sperling) Date: Sun, 21 Dec 2008 15:40:15 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> Message-ID: <20081221154015.GA6543@jack.stsp.name> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: > The right way to do this, IMNSHO is to not assume any particular > representation of TEXT values and create an implementation interface > that allows implementations of multiple text representations, much like > Rd and Wr don't make many assumptions about how data is actually stored > or retrieved. Such an interface may be needed for UTF-8 alone already, anyway, because within UTF-8 there is in some cases more than one way to store what amounts to the same data to a human user. In Subversion, from the beginning everyone agreed that the internal encoding for all strings would be UTF-8. Most Subversion APIs expect data in UTF-8. Strings (e.g. filenames) in the repository are stored in UTF-8, etc. Great! Will work in all countries! Right? Yes, but not on all operating systems if you're not careful! It did not occur to anyone at the time that there are characters which in UTF-8 have more than one representation (codepoints) in a byte stream. For example, an u with umlaut can be encoded as two bytes or a single byte: 2 bytes: [u | the previous character has an umlaut ] This is called "normal form decomposed". 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) This is called "normal form composed". If you want to be portable, as CM3 and Subversion both try to be, you have to consider that some operating systems may return your filenames in a different encoding then you stored it in: -------- Accepts Gives back MacOS X * NFD(*) Linux * Windows * Others ? ? *) There are some remarks to be made regarding full or partial NFD here, but the essential thing is: If you send in NFC, don't expect it back! -------- quoted from: http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames which is worth a read for more details if you're interested. In Subversion, this is a real problem for Mac users, because two filenames which only differ in their NFC/NFD encoding look exactly the same to the user (an u umlaut is printed), while the byte streams do not match ("We're sorry, but your file x does not exist in the repository!", where x looks just like a file that is clearly visible in the repository listing :) Subversion's problem now is that there are repositories out there using filenames in either NFC, NFD, or mixed, and there is no good way to reconcile the mess while staying backwards compatible with existing clients, servers, working copies and repositories. So Mac users are told to only use ASCII characters in their filenames to prevent the problem (many users, especially users who are not programmers, who store their photos or their entire home directory or whatever in Subversion, are not happy about this). This problem may not matter as much in case of CM3, but anyone implementing UTF-8 support for CM3 should be aware of this issue and not repeat the mistake the Subversion developers made at the time! With UTF-8, do not rely on a filename to retain its encoding as you passed it to the OS when requesting the filename from the OS again. CM3 should pick either NFD or NFC as internal UTF-8 encoding, for filenames only, or for all strings, whichever makes more sense. And then stick to it, converting input/output as needed. Abstracting this problem away using a nice interface would probably be the cleanest solution. Stefan From jay.krell at cornell.edu Sun Dec 21 18:51:14 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:51:14 +0000 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: This is an issue with Unicode, not UTF8, right? ie: a 16 or 20 or 32 bit encoding has the same problem, right? search the web for "unicode precomposed": http://en.wikipedia.org/wiki/Precomposed_character http://wikisource.org/wiki/Unicode_precomposed_characters or "unicode precomposed apple": http://developer.apple.com/jp/qa/qa2001/qa1235.html http://developer.apple.com/qa/qa2001/qa1235.html "When working within Mac OS you will find yourself using a mixture of precomposed and decomposed Unicode. For example, HFS Plus converts all file names to decomposed Unicode, while Macintosh keyboards generally produce precomposed Unicode. This isn't a problem as long as you use system-provided APIs to process text. Apple's APIs correctly handle both precomposed and decomposed Unicode. However, you may need to convert to precomposed Unicode when you interact with other platforms. For example, the following are all valid reasons why you might want to convert to precomposed Unicode. If you implement a network protocol which is defined to use precomposed Unicode. When creating a cross-platform file (or volume) whose specification dictates precomposed Unicode. If you incorporate a large body of cross-platform code into your application, where that code is expecting precomposed Unicode. " http://www.unicode.org/unicode/reports/tr15/index.html I need to read these.. - Jay> Date: Sun, 21 Dec 2008 15:40:15 +0000> From: stsp at elego.de> To: darko at darko.org> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] This disgusting TEXT business> > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote:> > The right way to do this, IMNSHO is to not assume any particular > > representation of TEXT values and create an implementation interface > > that allows implementations of multiple text representations, much like > > Rd and Wr don't make many assumptions about how data is actually stored > > or retrieved.> > Such an interface may be needed for UTF-8 alone already, anyway,> because within UTF-8 there is in some cases more than one way> to store what amounts to the same data to a human user.> > In Subversion, from the beginning everyone agreed that the internal> encoding for all strings would be UTF-8. Most Subversion APIs expect> data in UTF-8. Strings (e.g. filenames) in the repository are stored> in UTF-8, etc. Great! Will work in all countries! Right?> > Yes, but not on all operating systems if you're not careful!> It did not occur to anyone at the time that there are characters> which in UTF-8 have more than one representation (codepoints) in a> byte stream. For example, an u with umlaut can be encoded as two> bytes or a single byte:> > 2 bytes: [u | the previous character has an umlaut ]> This is called "normal form decomposed".> > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :)> This is called "normal form composed".> > If you want to be portable, as CM3 and Subversion both try to be,> you have to consider that some operating systems may return your> filenames in a different encoding then you stored it in:> > --------> Accepts Gives back> MacOS X * NFD(*)> Linux * > Windows * > Others ? ?> > > *) There are some remarks to be made regarding full or partial> NFD here, but the essential thing is: If you send in NFC, don't> expect it back!> -------- quoted from:> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames> which is worth a read for more details if you're interested.> > In Subversion, this is a real problem for Mac users, because> two filenames which only differ in their NFC/NFD encoding> look exactly the same to the user (an u umlaut is printed),> while the byte streams do not match ("We're sorry, but your> file x does not exist in the repository!", where x looks just> like a file that is clearly visible in the repository listing :)> > Subversion's problem now is that there are repositories out> there using filenames in either NFC, NFD, or mixed, and there> is no good way to reconcile the mess while staying backwards> compatible with existing clients, servers, working copies and> repositories. So Mac users are told to only use ASCII characters> in their filenames to prevent the problem (many users, especially> users who are not programmers, who store their photos or their> entire home directory or whatever in Subversion, are not happy> about this).> > This problem may not matter as much in case of CM3, but anyone> implementing UTF-8 support for CM3 should be aware of this issue> and not repeat the mistake the Subversion developers made at the> time! With UTF-8, do not rely on a filename to retain its encoding> as you passed it to the OS when requesting the filename from the> OS again.> > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for> filenames only, or for all strings, whichever makes more sense.> And then stick to it, converting input/output as needed.> > Abstracting this problem away using a nice interface would probably> be the cleanest solution.> > Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Dec 21 18:57:00 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 21 Dec 2008 17:57:00 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <465668.39592.qm@web28601.mail.ukl.yahoo.com> References: <465668.39592.qm@web28601.mail.ukl.yahoo.com> Message-ID: The real world is unsafe.Safe areas are created by unsafe code, which must be, to a certain extent, bug free, in order to provide as safe as advertised. I don't believe what I added is any more untrusted/unsafe than what was there before.Well, maybe in a minor way. m3core has "lots" of "external" functions, which are called by "lots" of functions at higher levels..ok, maybe not "lots", but if you look at the m3core/src/unix directories that I have worked on -- cygwin, openbsd, amd64_linux, linux-common -- this is a fairly minimal set of stuff that is being used by preexisting code. A LITTLE bit I added beyond what was there. If you look at all the other directories, you'll find lots more external stuff, most of which is not used within the CM3 codebase (but could be used by folks outside). It is true I may have introduced external where it wasn't previously, but I don't think it matters.It is prevalent one level down. Moving some of it one level up is ok I believe. Non-Posix does potentially have a place in .c files.It depends.Basically, as I see things, a small amount of #ifdef is ok.A lot of #ifdef becomes hard to read, and at some point, you split the files into separate files, "no" #ifdefs -- or conceptually each file can be viewed as wrapped in an #ifdef or other platform detection.You need #ifdef in any C that uses 64 bit integers, for example. Current Microsoft compilers do support "long long" but __int64 supporg goes way back, and __int64 is a better name anyway. What is the 128 bit integer type to be called -- "__int128" or "long long long". The names "char", "short", "int", "long", "long long" are mostly just funny mistaken names. The right names to provide are explicitly sized, int8, uint8, int16, uint16, int32, uint32, int64, uint64, and pointer sized types such as "size_t" and "diff_t". Modula-3 I think is reasonable here, since it's primary integral type is pointer sized, and it has "bits x for ...", though the other types probably ought to be built in and blessed somewhere, not just in the Csupport library section. Not sure what "Posix" means, since it might incorporate by-reference ANSI C. - Jay Date: Sat, 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3devel at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Dear all:I'm glad that Jay and all the people around are making contributions, so we can claim that as some day there were a good number of supported platforms nowadays we also have.However I have a question, but its more a suggestion, the modification Tony is referring is not as standard as we would like since the type safety is affected with the change introduced allowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix, it's an implementation issue of the CM3 (IMHO it should warn) compiler, so we may want to mark as UNSAFE INTERFACE, however doing that I think some clients could not claim any more they are safe so that "fix" is not feasible.Can you push to a lower level (as in the UNSAFE FilePosix implementation module before committing the change) or make it available in other Interface already available, why do we need that change anyway? Also IMHO we should leave out what is not POSIX (we should not care for WIN64 although I know is needed by the C compiler preprocessor) out of that file.Although I don't have the formal specification or program, but I can remember libm3 code was specified and mechanically checked with the Extended Static Checker Modula-3 (which is not sound also, that is could be cheated, see http://www.researchchannel.org/prog/displayevent.aspx?rID=2761&fID=345 ), that would be a good reason for keeping as safe as possible and close to historical interfaces and just rework implementations (even if they are not safe) as needed. I know that implementation is not safe but at least having it well enough implemented the ESC/M3 checker and the compiler would tell us the library should work Ok and is safe and (hopefully) without the errors detected by the ESC as we care for the details in the UNSAFE implementation.There are any special interfaces we should have as "standard" in the lower level runtime (hard enough question I know), can we give some hints/ideas on that?Thanks in advance--- El vie, 19/12/08, Jay escribi?: De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3devel" , "Tony" Fecha: viernes, 19 diciembre, 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is ok now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I only checked SOLgnu. I will check SOLsun. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Dec 22 15:20:12 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 09:20:12 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1229855598.18256.168.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <494D3B89.3020802@gmx.de> <1229855598.18256.168.camel@faramir.m3w.org> Message-ID: <20081222142011.GB28224@topoi.pooq.com> On Sun, Dec 21, 2008 at 11:33:18AM +0100, Dragi?a Duri? wrote: > Most important argument for UTF-8 - IMO - would be it's completeness. > 2byte WIDECHAR is not complete. Period. Be it mahhjong tile or some > obscure Chinese glyph - 16 bytes is not enough. Would we do some 20bit > storage magic instead? Well, 20 bits would certianly fit inside 16 bytes, :-) -- hendrik From darko at darko.org Sun Dec 21 22:07:21 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 06:07:21 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081221154015.GA6543@jack.stsp.name> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> Message-ID: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> The way to handle this would be to require an iterate function on each text representation which successively returned the next character in a string as a UNICHAR with defined characteristics. That would allow for comparison and transfer of strings. The way I see it, TEXT should be revealed to be a subtype of the "TextObj" object that has methods that basically mirror the Text interface with some additions. The existing text representation code is adapted to this object (fairly simple) and new representations subclass that object. A new function is added to text interface that allows the user to specify the default representation used when creating new strings. This would be the typecode of the object to allocate, required to be a subtype of TextObj of course. Some TextObj methods would be required to be implemented by the representation but most could be handled by the ancestor. Many of the methods could be implemented by the encoding for more efficient handling, for instance when the text being compared or concatenated is the same representation. UNICHAR = ; TextObj = OBJECT METHODS length(t: T): CARDINAL; (* required to be implemented by the encoding *) empty(t: T): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(t: T): BOOLEAN; (* required - meaning containing anything other than CHAR values *) next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index otherwise meaningless and private. Seek allows skipping forward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(t, u: T): BOOLEAN; compare(t1, t2: T): [-1..1]; cat(t, u: T): T; sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(t: T): Word.T; getChar(t: T; i: CARDINAL): CHAR; getWideChar(t: T; i: CARDINAL): WIDECHAR; getUniChar(t: T; i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWIdeChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(t: T; c: CHAR; start := 0): INTEGER; findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; Additionally the Text and a couple of other interfaces (eg Rd, Wr) would need to be expanded to handle UniChar. On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >> The right way to do this, IMNSHO is to not assume any particular >> representation of TEXT values and create an implementation interface >> that allows implementations of multiple text representations, much >> like >> Rd and Wr don't make many assumptions about how data is actually >> stored >> or retrieved. > > Such an interface may be needed for UTF-8 alone already, anyway, > because within UTF-8 there is in some cases more than one way > to store what amounts to the same data to a human user. > > In Subversion, from the beginning everyone agreed that the internal > encoding for all strings would be UTF-8. Most Subversion APIs expect > data in UTF-8. Strings (e.g. filenames) in the repository are stored > in UTF-8, etc. Great! Will work in all countries! Right? > > Yes, but not on all operating systems if you're not careful! > It did not occur to anyone at the time that there are characters > which in UTF-8 have more than one representation (codepoints) in a > byte stream. For example, an u with umlaut can be encoded as two > bytes or a single byte: > > 2 bytes: [u | the previous character has an umlaut ] > This is called "normal form decomposed". > > 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) > This is called "normal form composed". > > If you want to be portable, as CM3 and Subversion both try to be, > you have to consider that some operating systems may return your > filenames in a different encoding then you stored it in: > > -------- > Accepts Gives back > MacOS X * NFD(*) > Linux * > Windows * > Others ? ? > > > *) There are some remarks to be made regarding full or partial > NFD here, but the essential thing is: If you send in NFC, don't > expect it back! > -------- quoted from: > http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames > which is worth a read for more details if you're interested. > > In Subversion, this is a real problem for Mac users, because > two filenames which only differ in their NFC/NFD encoding > look exactly the same to the user (an u umlaut is printed), > while the byte streams do not match ("We're sorry, but your > file x does not exist in the repository!", where x looks just > like a file that is clearly visible in the repository listing :) > > Subversion's problem now is that there are repositories out > there using filenames in either NFC, NFD, or mixed, and there > is no good way to reconcile the mess while staying backwards > compatible with existing clients, servers, working copies and > repositories. So Mac users are told to only use ASCII characters > in their filenames to prevent the problem (many users, especially > users who are not programmers, who store their photos or their > entire home directory or whatever in Subversion, are not happy > about this). > > This problem may not matter as much in case of CM3, but anyone > implementing UTF-8 support for CM3 should be aware of this issue > and not repeat the mistake the Subversion developers made at the > time! With UTF-8, do not rely on a filename to retain its encoding > as you passed it to the OS when requesting the filename from the > OS again. > > CM3 should pick either NFD or NFC as internal UTF-8 encoding, for > filenames only, or for all strings, whichever makes more sense. > And then stick to it, converting input/output as needed. > > Abstracting this problem away using a nice interface would probably > be the cleanest solution. > > Stefan From hendrik at topoi.pooq.com Mon Dec 22 16:54:38 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 10:54:38 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <20081222155438.GA28375@topoi.pooq.com> On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: > The way to handle this would be to require an iterate function on each > text representation which successively returned the next character in > a string as a UNICHAR with defined characteristics. That would allow > for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some TextObj > methods would be required to be implemented by the representation but > most could be handled by the ancestor. Many of the methods could be > implemented by the encoding for more efficient handling, for instance > when the text being compared or concatenated is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) How is T related to TextObj? Does this return the number of characters of the number of bytes occupied by the stinrg? > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) Does this require a scan of the string to determine whether any wide characters are actually present, or does it just indicate whether the data representation used in this string is capable of handling wide characters? > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the next > logical character in char and returns true, or false if the char > doesn't exist. Index otherwise meaningless and private. Seek allows > skipping forward a number of characters. *) Do we need a colon after the second VAR? This seems intended for an implementation where index is a byte offset. Is it possible to copy index so as to start a new interatin where at a saved point? > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) What stack? > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) length in bytes or characters? > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; I'd really like functions that provide access to the underlying UTF8 encoding on a byte-by-byte basis. Often that's the most efficient way, and the simplest, to process UTF8-encoded data, and UTF8 had been designed with this in mind. > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > while we are at it, let's include byte access as well. Treating bytes as integers between 0 and 255 would work. -- hendrik Note: when I'm processing UTF8 data, I'm often parsing it and inserting other tokens (which are not UTF8 characters) into the data stream as well. It is convenient to use negative numbers for this, clearly distinguished from Unicode codepints, which are positive. So making characters availabla as positive integers would work cleanly with this. From darko at darko.org Mon Dec 22 01:07:39 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:07:39 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081222155438.GA28375@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> Message-ID: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> On 23/12/2008, at 12:54 AM, hendrik at topoi.pooq.com wrote: > On Mon, Dec 22, 2008 at 06:07:21AM +0900, Darko wrote: >> The way to handle this would be to require an iterate function on >> each >> text representation which successively returned the next character in >> a string as a UNICHAR with defined characteristics. That would allow >> for comparison and transfer of strings. >> >> The way I see it, TEXT should be revealed to be a subtype of the >> "TextObj" object that has methods that basically mirror the Text >> interface with some additions. The existing text representation code >> is adapted to this object (fairly simple) and new representations >> subclass that object. A new function is added to text interface that >> allows the user to specify the default representation used when >> creating new strings. This would be the typecode of the object to >> allocate, required to be a subtype of TextObj of course. Some TextObj >> methods would be required to be implemented by the representation but >> most could be handled by the ancestor. Many of the methods could be >> implemented by the encoding for more efficient handling, for instance >> when the text being compared or concatenated is the same >> representation. >> >> UNICHAR = ; >> >> TextObj = OBJECT METHODS >> >> length(t: T): CARDINAL; (* required to be implemented by the >> encoding *) > > How is T related to TextObj? Does this return the number of > characters > of the number of bytes occupied by the stinrg? Sorry, T = TEXT. It returns the logical number of characters according to some definition, which I imagine would be Unicode NFC (Normalised Form, Canonical Composition). > > >> empty(t: T): BOOLEAN; (* required to be implemented by the encoding >> *) >> hasWideChars(t: T): BOOLEAN; (* required - meaning containing >> anything other than CHAR values *) > > Does this require a scan of the string to determine whether any wide > characters are actually present, or does it just indicate whether the > data representation used in this string is capable of handling wide > characters? No to both. It doesn't require a scan, you can cache this flag. It's so you can possibly be more efficient when you know that you only have to deal with CHAR values. > > >> next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): >> BOOLEAN; (* required - start iterating with index=0, returns the next >> logical character in char and returns true, or false if the char >> doesn't exist. Index otherwise meaningless and private. Seek allows >> skipping forward a number of characters. *) > > Do we need a colon after the second VAR? No, it's a typo. > This seems intended for an implementation where index is a byte > offset. > Is it possible to copy index so as to start a new interatin where at a > saved point? Yes. > > >> getData(): ADDRESS; (* required - get the raw data, only valid >> while on the stack *) > > What stack? The execution stack. It might be a REFANY so would change if it didn't appear in the stack. > > >> setData(adr: ADDRESS; length: CARDINAL); (* required - set the data >> and length of the encoded data *) > > length in bytes or characters? Bytes, it's the length of the raw data and has no logical interpretation. > > >> >> (* the remaining methods are optional for the encoding >> implementation *) >> equal(t, u: T): BOOLEAN; >> compare(t1, t2: T): [-1..1]; >> cat(t, u: T): T; >> sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; >> hash(t: T): Word.T; >> getChar(t: T; i: CARDINAL): CHAR; >> getWideChar(t: T; i: CARDINAL): WIDECHAR; >> getUniChar(t: T; i: CARDINAL): UNICHAR; > > I'd really like functions that provide access to the underlying UTF8 > encoding on a byte-by-byte basis. Often that's the most efficient > way, > and the simplest, to process UTF8-encoded data, and UTF8 had been > designed with this in mind. If you know the text is encoded as UTF-8 then you can get the raw data as above. > > >> setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); >> setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); >> setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); >> fromChar(ch: CHAR): T; >> fromWIdeChar(ch: WIDECHAR): T; >> fromUniChar(ch: UNICHAR): T; >> fromChars(READONLY a: ARRAY OF CHAR): T; >> fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; >> fromUniChars(READONLY a: ARRAY OF UNICHAR): T; >> findChar(t: T; c: CHAR; start := 0): INTEGER; >> findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; >> findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; >> findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; >> findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; >> findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; >> END; >> >> Additionally the Text and a couple of other interfaces (eg Rd, Wr) >> would need to be expanded to handle UniChar. >> > > while we are at it, let's include byte access as well. Treating bytes > as integers between 0 and 255 would work. Can't see the point since you can ORD a char value to get the same. > > > -- hendrik > > Note: when I'm processing UTF8 data, I'm often parsing it and > inserting > other tokens (which are not UTF8 characters) into the data stream as > well. It is convenient to use negative numbers for this, clearly > distinguished from Unicode codepints, which are positive. So making > characters availabla as positive integers would work cleanly with > this. > > From darko at darko.org Mon Dec 22 01:14:16 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 09:14:16 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> A couple of corrections: - T = TEXT and TextObj <: T so it should read TextObj = T OBJECT - the cat method is required for the encoding implementation (helps to actually be able to create strings) - the CARDINAL in the next method should be INTEGER to enable searching backwards (findCharR) and should probably be named 'iter'. On 22/12/2008, at 6:07 AM, Darko wrote: > The way to handle this would be to require an iterate function on > each text representation which successively returned the next > character in a string as a UNICHAR with defined characteristics. > That would allow for comparison and transfer of strings. > > The way I see it, TEXT should be revealed to be a subtype of the > "TextObj" object that has methods that basically mirror the Text > interface with some additions. The existing text representation code > is adapted to this object (fairly simple) and new representations > subclass that object. A new function is added to text interface that > allows the user to specify the default representation used when > creating new strings. This would be the typecode of the object to > allocate, required to be a subtype of TextObj of course. Some > TextObj methods would be required to be implemented by the > representation but most could be handled by the ancestor. Many of > the methods could be implemented by the encoding for more efficient > handling, for instance when the text being compared or concatenated > is the same representation. > > UNICHAR = ; > > TextObj = OBJECT METHODS > > length(t: T): CARDINAL; (* required to be implemented by the > encoding *) > empty(t: T): BOOLEAN; (* required to be implemented by the encoding > *) > hasWideChars(t: T): BOOLEAN; (* required - meaning containing > anything other than CHAR values *) > next(VAR index: INTEGER; VAR: char: UNICHAR; seek: CARDINAL := 1): > BOOLEAN; (* required - start iterating with index=0, returns the > next logical character in char and returns true, or false if the > char doesn't exist. Index otherwise meaningless and private. Seek > allows skipping forward a number of characters. *) > getData(): ADDRESS; (* required - get the raw data, only valid > while on the stack *) > setData(adr: ADDRESS; length: CARDINAL); (* required - set the data > and length of the encoded data *) > > (* the remaining methods are optional for the encoding > implementation *) > equal(t, u: T): BOOLEAN; > compare(t1, t2: T): [-1..1]; > cat(t, u: T): T; > sub(t: T; start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; > hash(t: T): Word.T; > getChar(t: T; i: CARDINAL): CHAR; > getWideChar(t: T; i: CARDINAL): WIDECHAR; > getUniChar(t: T; i: CARDINAL): UNICHAR; > setChars(VAR a: ARRAY OF CHAR; t: T; start: CARDINAL := 0); > setWideChars(VAR a: ARRAY OF WIDECHAR; t: T; start: CARDINAL := 0); > setUniChars(VAR a: ARRAY OF UNICHAR; t: T; start: CARDINAL := 0); > fromChar(ch: CHAR): T; > fromWIdeChar(ch: WIDECHAR): T; > fromUniChar(ch: UNICHAR): T; > fromChars(READONLY a: ARRAY OF CHAR): T; > fromWIdeChars(READONLY a: ARRAY OF WIDECHAR): T; > fromUniChars(READONLY a: ARRAY OF UNICHAR): T; > findChar(t: T; c: CHAR; start := 0): INTEGER; > findWideChar(t: T; c: WIDECHAR; start := 0): INTEGER; > findUniChar(t: T; c: UNICHAR; start := 0): INTEGER; > findCharR(t: T; c: CHAR; start := LAST(INTEGER)): INTEGER; > findWideCharR(t: T; c: WIDECHAR; start := LAST(INTEGER)): INTEGER; > findUniCharR(t: T; c: UNICHAR; start := LAST(INTEGER)): INTEGER; > END; > > Additionally the Text and a couple of other interfaces (eg Rd, Wr) > would need to be expanded to handle UniChar. > > > > On 22/12/2008, at 12:40 AM, Stefan Sperling wrote: > >> On Sun, Dec 21, 2008 at 08:08:57AM +0900, Darko wrote: >>> The right way to do this, IMNSHO is to not assume any particular >>> representation of TEXT values and create an implementation interface >>> that allows implementations of multiple text representations, much >>> like >>> Rd and Wr don't make many assumptions about how data is actually >>> stored >>> or retrieved. >> >> Such an interface may be needed for UTF-8 alone already, anyway, >> because within UTF-8 there is in some cases more than one way >> to store what amounts to the same data to a human user. >> >> In Subversion, from the beginning everyone agreed that the internal >> encoding for all strings would be UTF-8. Most Subversion APIs expect >> data in UTF-8. Strings (e.g. filenames) in the repository are stored >> in UTF-8, etc. Great! Will work in all countries! Right? >> >> Yes, but not on all operating systems if you're not careful! >> It did not occur to anyone at the time that there are characters >> which in UTF-8 have more than one representation (codepoints) in a >> byte stream. For example, an u with umlaut can be encoded as two >> bytes or a single byte: >> >> 2 bytes: [u | the previous character has an umlaut ] >> This is called "normal form decomposed". >> >> 1 byte [u umlaut] (i.e. ? if you can see this on your terminal :) >> This is called "normal form composed". >> >> If you want to be portable, as CM3 and Subversion both try to be, >> you have to consider that some operating systems may return your >> filenames in a different encoding then you stored it in: >> >> -------- >> Accepts Gives back >> MacOS X * NFD(*) >> Linux * >> Windows * >> Others ? ? >> >> >> *) There are some remarks to be made regarding full or partial >> NFD here, but the essential thing is: If you send in NFC, don't >> expect it back! >> -------- quoted from: >> http://svn.collab.net/repos/svn/trunk/notes/unicode-composition-for-filenames >> which is worth a read for more details if you're interested. >> >> In Subversion, this is a real problem for Mac users, because >> two filenames which only differ in their NFC/NFD encoding >> look exactly the same to the user (an u umlaut is printed), >> while the byte streams do not match ("We're sorry, but your >> file x does not exist in the repository!", where x looks just >> like a file that is clearly visible in the repository listing :) >> >> Subversion's problem now is that there are repositories out >> there using filenames in either NFC, NFD, or mixed, and there >> is no good way to reconcile the mess while staying backwards >> compatible with existing clients, servers, working copies and >> repositories. So Mac users are told to only use ASCII characters >> in their filenames to prevent the problem (many users, especially >> users who are not programmers, who store their photos or their >> entire home directory or whatever in Subversion, are not happy >> about this). >> >> This problem may not matter as much in case of CM3, but anyone >> implementing UTF-8 support for CM3 should be aware of this issue >> and not repeat the mistake the Subversion developers made at the >> time! With UTF-8, do not rely on a filename to retain its encoding >> as you passed it to the OS when requesting the filename from the >> OS again. >> >> CM3 should pick either NFD or NFC as internal UTF-8 encoding, for >> filenames only, or for all strings, whichever makes more sense. >> And then stick to it, converting input/output as needed. >> >> Abstracting this problem away using a nice interface would probably >> be the cleanest solution. >> >> Stefan > From hendrik at topoi.pooq.com Mon Dec 22 21:10:36 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Mon, 22 Dec 2008 15:10:36 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <20081222155438.GA28375@topoi.pooq.com> <29720505-D994-4091-97BF-B0CE64A62AE7@darko.org> Message-ID: <20081222201036.GC28728@topoi.pooq.com> On Mon, Dec 22, 2008 at 09:07:39AM +0900, Darko wrote: > > > > > >> getData(): ADDRESS; (* required - get the raw data, only valid > >>while on the stack *) > > > >What stack? > > The execution stack. It might be a REFANY so would change if it didn't > appear in the stack. I must be misunderstanding something; this seems perfectly obvious to you, but opaque to me. > >> > > > >while we are at it, let's include byte access as well. Treating bytes > >as integers between 0 and 255 would work. > > Can't see the point since you can ORD a char value to get the same. Not unless it's *specified* that the raw data structure *is* a bunch of CHARs, and that CHARs are single bytes. This whole discussion is occasioned by the discovery that characters are no longer single bytes, but strings of bytes. -- hendrik From darko at darko.org Mon Dec 22 03:52:38 2008 From: darko at darko.org (Darko) Date: Mon, 22 Dec 2008 11:52:38 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <55C6A3B5-EF28-45EB-9BAE-F44580A9EE14@darko.org> Message-ID: <5A505AA9-97B5-4B20-878C-0FDC5359B8A7@darko.org> Ah the perils of copy and paste early in the morning. Here is a corrected version without extraneous method parameters. UNICHAR = ; T = TEXT; T <: TextObj; TextObj = OBJECT METHODS length(): CARDINAL; (* required to be implemented by the encoding *) empty(): BOOLEAN; (* required to be implemented by the encoding *) hasWideChars(): BOOLEAN; (* required - meaning containing anything other than CHAR values *) cat(u: T): T; (* required to be implemented by the encoding *) iter(VAR index: INTEGER; VAR char: UNICHAR; seek: INTEGER := 1): BOOLEAN; (* required - start iterating with index=0, returns the next logical character in char and returns true, or false if the char doesn't exist. Index is otherwise meaningless and private. Seek allows skipping forward or backward a number of characters. *) getData(): ADDRESS; (* required - get the raw data, only valid while on the stack *) setData(adr: ADDRESS; length: CARDINAL); (* required - set the data and length of the encoded data *) (* the remaining methods are optional for the encoding implementation *) equal(u: T): BOOLEAN; compare(u: T): [-1..1]; sub(start: CARDINAL;length: CARDINAL := LAST(CARDINAL)): T; hash(): Word.T; getChar(i: CARDINAL): CHAR; getWideChar(i: CARDINAL): WIDECHAR; getUniChar(i: CARDINAL): UNICHAR; setChars(VAR a: ARRAY OF CHAR; start: CARDINAL := 0); setWideChars(VAR a: ARRAY OF WIDECHAR; start: CARDINAL := 0); setUniChars(VAR a: ARRAY OF UNICHAR; start: CARDINAL := 0); fromChar(ch: CHAR): T; fromWideChar(ch: WIDECHAR): T; fromUniChar(ch: UNICHAR): T; fromChars(READONLY a: ARRAY OF CHAR): T; fromWideChars(READONLY a: ARRAY OF WIDECHAR): T; fromUniChars(READONLY a: ARRAY OF UNICHAR): T; findChar(c: CHAR; start := 0): INTEGER; findWideChar(c: WIDECHAR; start := 0): INTEGER; findUniChar(c: UNICHAR; start := 0): INTEGER; findCharR(c: CHAR; start := LAST(INTEGER)): INTEGER; findWideCharR(c: WIDECHAR; start := LAST(INTEGER)): INTEGER; findUniCharR(c: UNICHAR; start := LAST(INTEGER)): INTEGER; END; PROCEDURE SetTextEnc(tc: INTEGER); (* set the encoding object to use from its typecode, must be subtype of TextObj *) PROCEDURE GetTextEnc(): INTEGER: (* returns the typecode of the current encoding object in use *) From hosking at cs.purdue.edu Mon Dec 22 21:56:50 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 23 Dec 2008 07:56:50 +1100 Subject: [M3devel] Fwd: Output from "cron" command References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: > From: Tony Hosking > Date: 23 December 2008 00:04:20 GMT+11:00 > To: hosking at cs.purdue.edu > Subject: Output from "cron" command > > Your "cron" job on niagara.cs.purdue.edu > $HOME/cm3/scripts/regression/cron.sh > > produced the following output: > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara release-build" > > creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:30:06 -- checkout in progress. > [start checkout 2008.12.22 06:30:10] > cd /tmp/build-cm3-20081222-063006-5jaOwe/build > cvs return value: 0 > [end checkout 2008.12.22 06:49:53] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 06:49:55 -- compile in progress. > [start compile 2008.12.22 06:49:55] > compile return value: 0 > [end compile 2008.12.22 06:55:09] > COMPILE_RETURN = 1 > *** COMPILE FAILED > removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > TESTHOSTNAME=niagara > WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 > LASTREL=5.4.0 > INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 > INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok > INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok > INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current > CM3_OSTYPE=POSIX > CM3_TARGET=SOLgnu > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > CM3CVSSERVER=birch.elegosoft.com > CM3CVSROOT=birch.elegosoft.com:/usr/cvs > BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz > BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz > testing ssh birch.elegosoft.com.. > ssh birch.elegosoft.com ok > Building cm3. > Tinderbox Tree: "cm3" > Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" > > creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt > > --- > > checkout, compile and test of cm3 ... > 2008.12.22 06:57:16 -- checkout in progress. > [start checkout 2008.12.22 06:57:18] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > cvs return value: 0 > [end checkout 2008.12.22 07:16:39] > CHECKOUT_RETURN = 0 > -- > 2008.12.22 07:16:41 -- compile in progress. > [start compile 2008.12.22 07:16:41] > compile return value: 0 > [end compile 2008.12.22 08:02:05] > COMPILE_RETURN = 0 > 2008.12.22 08:02:22 -- tests in progress. > [start run-tests 2008.12.22 08:02:22] > cd /tmp/build-cm3-20081222-065716-Oha4Uk/build > [end run-tests 2008.12.22 08:02:22] > TESTS_RETURN = 0 > 2008.12.22 08:02:22 -- checkout, compile and test run done. > > --- > > removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... > cleaning CM3 workspaces... > /homes/hosking/work/cm3-ws/niagara-* > > cleaning regression test log files... > /homes/hosking/tmp/cm3/niagara/cm3-rlog-* > > cleaning m3test log files... > /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr > > /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract > > cleaning snapshot files... > /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz > > cleaning package reports... > /tmp/cm3-pkg-report-SOLgnu-*.html > > done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 04:06:37 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Mon, 22 Dec 2008 21:06:37 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> References: <200812210002.mBL02B2N009216@camembert.async.caltech.edu> Message-ID: <495055BD.2010009@wichita.edu> Mika Nystrom wrote: > Note to self and others: m3gdb is not completely useful here. If > you look at the structure of a TextCat.T, you see that fields a and > b are TEXTs, but m3gdb doesn't really let you look inside them, > since, well, they are TEXTs and printed as such. (You don't > get to see the whole gory mess of pointers.) Is there a way to > turn off this (normally very helpful) m3gdb feature? Yes. Perhaps a couple of years ago, I added a /k format letter to m3gdb for printing TEXT values showing the internal object, rather than the characters in quotes. At the time, I only announced this in the CVS checkin logs (which show up in the m3commit mailing list), but recently, I added a good-sized bit of collected documentation for m3gdb, which describes this in more detail. From: http://modula3.elegosoft.com/cm3/doc/help/m3gdb/m3gdb-onepage.html#id2517604 ----------------------------------------------------------------------------- Normally, m3gdb displays TEXT values in Modula-3 lexical syntax, with double quotes, escape sequences, and, if appropriate, a leading 'W'. However, the /k option in a print command will cause it instead to display the TEXT value's internal data structure. For PM3 et. al., this is a traced reference to an open array of characters. For CM3, this will be one of the several object subtypes of TEXT. m3gdb properly recognizes and displays values of CM3 type TextLiteral.T. Here, it takes the length of the text from the appropriate field, instead of from the declared type, which contains a fixed array whose length is amost always far too long. Even without specifying the /k option, if you happen to know what the internal representation is, you can apply appropriate operators to it, e.g., dereferencing, field selection, subscripting, or even method calls. ----------------------------------------------------------------------------- Examples: (m3gdb) p T6 $15 = (*16_b740e1dc*) "TextLiteralabcdefghij" (m3gdb) p/k T6 $16 = (*16_b740e1dc*) OBJECT a = 16_0804a670; b = 16_b740e0c8; a_len = 11; b_len = 10; a_or_b_wide = FALSE; END (m3gdb) p/k T6.a $17 = (*16_0804a670*) (*TextLiteral.T*) cnt = 11; buf = "TextLiteral" END (m3gdb) p/k T6.b $18 = (*16_b740e0c8*) OBJECT len = 10; contents = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', '\000', '\000', '\000', '\000', '\000', '\000'}; END ----------------------------------------------------------------------------- It would be nicer if m3gdb also always printed the name of the type of the object, but I haven't figured out what to do here in general, because an object type can have no name, or more than one name. But I see I am now printing (*TextLiteral.T*) as a special case, so the other CM3 subtypes of TEXT could be handled the same way. 'Nother item for my long m3gdb to-do list. Rodney Bates > > Mika > > From jay.krell at cornell.edu Tue Dec 23 13:53:26 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 12:53:26 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: Drat. I fixed a likely problem (broken on other platforms) and I think "upgrade" succeeded, but then I messed things up -- no working gcc. It looks like I might not be home now till just after New Years due to unforseen circumstances. (very inclement weather, stuck elsewhere) Can you get a more specific error message? If so, I can checkin a guess late in the week. Otherwise, can it wait? - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Tue, 23 Dec 2008 07:56:50 +1100CC: m3devel at elegosoft.comSubject: [M3devel] Fwd: Output from "cron" command Jay, things are still failing on my SOLgnu installation after your changes. Did you ever try it on yours? Happy holidays to all! Begin forwarded message: From: Tony Hosking Date: 23 December 2008 00:04:20 GMT+11:00 To: hosking at cs.purdue.edu Subject: Output from "cron" command Your "cron" job on niagara.cs.purdue.edu$HOME/cm3/scripts/regression/cron.shproduced the following output:TESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara release-build"creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:30:06 -- checkout in progress.[start checkout 2008.12.22 06:30:10]cd /tmp/build-cm3-20081222-063006-5jaOwe/buildcvs return value: 0[end checkout 2008.12.22 06:49:53]CHECKOUT_RETURN = 0--2008.12.22 06:49:55 -- compile in progress.[start compile 2008.12.22 06:49:55]compile return value: 0[end compile 2008.12.22 06:55:09]COMPILE_RETURN = 1*** COMPILE FAILEDremoving build tree /tmp/build-cm3-20081222-063006-5jaOwe ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmlTESTHOSTNAME=niagaraWS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14LASTREL=5.4.0INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-okINSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-okINSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/currentCM3_OSTYPE=POSIXCM3_TARGET=SOLgnuBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgzCM3CVSSERVER=birch.elegosoft.comCM3CVSROOT=birch.elegosoft.com:/usr/cvsBINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgzBINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgztesting ssh birch.elegosoft.com..ssh birch.elegosoft.com okBuilding cm3.Tinderbox Tree: "cm3"Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt---checkout, compile and test of cm3 ...2008.12.22 06:57:16 -- checkout in progress.[start checkout 2008.12.22 06:57:18]cd /tmp/build-cm3-20081222-065716-Oha4Uk/buildcvs return value: 0[end checkout 2008.12.22 07:16:39]CHECKOUT_RETURN = 0--2008.12.22 07:16:41 -- compile in progress.[start compile 2008.12.22 07:16:41]compile return value: 0[end compile 2008.12.22 08:02:05]COMPILE_RETURN = 02008.12.22 08:02:22 -- tests in progress.[start run-tests 2008.12.22 08:02:22]cd /tmp/build-cm3-20081222-065716-Oha4Uk/build[end run-tests 2008.12.22 08:02:22]TESTS_RETURN = 02008.12.22 08:02:22 -- checkout, compile and test run done.---removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...cleaning CM3 workspaces.../homes/hosking/work/cm3-ws/niagara-*cleaning regression test log files.../homes/hosking/tmp/cm3/niagara/cm3-rlog-*cleaning m3test log files.../homes/hosking/tmp/cm3/niagara/m3tests-*.stdout/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr/homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extractcleaning snapshot files.../homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgzcleaning package reports.../tmp/cm3-pkg-report-SOLgnu-*.htmldone. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Dec 23 13:59:00 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 23 Dec 2008 13:59:00 +0100 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> Message-ID: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Quoting Tony Hosking : > Jay, things are still failing on my SOLgnu installation after your > changes. Did you ever try it on yours? According to tinderbox, all release builds fail due to undefined quake procedures: /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh -c "cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? " m3cc 1196 === package m3-sys/m3cc === 1197 +++ cm3 -build -override -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? +++ 1198 --- building in SOLgnu --- 1199 1200 NEXT "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: fs_exists 1201 fs_exists cannot be used unless we provide an official release before, because it has not been officially released yet. Some lastok-builds fail for other reasons. Has anybody time to fix these? Alternatively, we could back out the offending changes... > Happy holidays to all! Happy Christmas to all M3 users from me, too! Olaf > Begin forwarded message: > >> From: Tony Hosking >> Date: 23 December 2008 00:04:20 GMT+11:00 >> To: hosking at cs.purdue.edu >> Subject: Output from "cron" command >> >> Your "cron" job on niagara.cs.purdue.edu >> $HOME/cm3/scripts/regression/cron.sh >> >> produced the following output: >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara release-build" >> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:30:06 -- checkout in progress. >> [start checkout 2008.12.22 06:30:10] >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build >> cvs return value: 0 >> [end checkout 2008.12.22 06:49:53] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 06:49:55 -- compile in progress. >> [start compile 2008.12.22 06:49:55] >> compile return value: 0 >> [end compile 2008.12.22 06:55:09] >> COMPILE_RETURN = 1 >> *** COMPILE FAILED >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> TESTHOSTNAME=niagara >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14 >> LASTREL=5.4.0 >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0 >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current >> CM3_OSTYPE=POSIX >> CM3_TARGET=SOLgnu >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> CM3CVSSERVER=birch.elegosoft.com >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz >> testing ssh birch.elegosoft.com.. >> ssh birch.elegosoft.com ok >> Building cm3. >> Tinderbox Tree: "cm3" >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build" >> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt >> >> --- >> >> checkout, compile and test of cm3 ... >> 2008.12.22 06:57:16 -- checkout in progress. >> [start checkout 2008.12.22 06:57:18] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> cvs return value: 0 >> [end checkout 2008.12.22 07:16:39] >> CHECKOUT_RETURN = 0 >> -- >> 2008.12.22 07:16:41 -- compile in progress. >> [start compile 2008.12.22 07:16:41] >> compile return value: 0 >> [end compile 2008.12.22 08:02:05] >> COMPILE_RETURN = 0 >> 2008.12.22 08:02:22 -- tests in progress. >> [start run-tests 2008.12.22 08:02:22] >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build >> [end run-tests 2008.12.22 08:02:22] >> TESTS_RETURN = 0 >> 2008.12.22 08:02:22 -- checkout, compile and test run done. >> >> --- >> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ... >> cleaning CM3 workspaces... >> /homes/hosking/work/cm3-ws/niagara-* >> >> cleaning regression test log files... >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-* >> >> cleaning m3test log files... >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr >> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract >> >> cleaning snapshot files... >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz >> >> cleaning package reports... >> /tmp/cm3-pkg-report-SOLgnu-*.html >> >> done. -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 23 14:07:20 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 23 Dec 2008 13:07:20 +0000 Subject: [M3devel] Fwd: Output from "cron" command In-Reply-To: <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> References: <200812221304.mBMD4Kbw007895@niagara.cs.purdue.edu> <20081223135900.higeb052go0s0g8c@mail.elegosoft.com> Message-ID: ah. I thought for some reason that m3cc could use newer stuff, probably confusing with m3tests, and I wouldn't catch this as I back up to older versions rarely. That's /trivial/ to fix..though I'm still hampered by unusual real-world circumstance, should get relief by end of week (Friday), or maybe much earlier, but certainly during Friday day. The fix is to just provide one's own, like: if not defined("fs_exists") proc fs_exists(a) is return state(a, a) end end Thanks, - Jay> Date: Tue, 23 Dec 2008 13:59:00 +0100> From: wagner at elegosoft.com> To: m3devel at elegosoft.com> Subject: Re: [M3devel] Fwd: Output from "cron" command> > Quoting Tony Hosking :> > > Jay, things are still failing on my SOLgnu installation after your> > changes. Did you ever try it on yours?> > According to tinderbox, all release builds fail due to undefined> quake procedures:> > > /scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/scripts/pkgmap.sh > -c "cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? " m3cc> 1196 === package m3-sys/m3cc ===> 1197 +++ cm3 -build -override > -DROOT=?/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? +++> 1198 --- building in SOLgnu ---> 1199> 1200 NEXT > "/scratch/hosking/work/cm3-ws/niagara-2008-12-23-11-30-03/cm3/m3-sys/m3cc/src/m3makefile", line 293: quake runtime error: undefined variable: > fs_exists> 1201> > fs_exists cannot be used unless we provide an official release before,> because it has not been officially released yet.> > Some lastok-builds fail for other reasons.> > Has anybody time to fix these? Alternatively, we could back out the> offending changes...> > > Happy holidays to all!> > Happy Christmas to all M3 users from me, too!> > Olaf> > > Begin forwarded message:> >> >> From: Tony Hosking > >> Date: 23 December 2008 00:04:20 GMT+11:00> >> To: hosking at cs.purdue.edu> >> Subject: Output from "cron" command> >>> >> Your "cron" job on niagara.cs.purdue.edu> >> $HOME/cm3/scripts/regression/cron.sh> >>> >> produced the following output:> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-30-04> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara release-build"> >>> >> creating log file /tmp/build-cm3-20081222-063006-5jaOwe/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:30:06 -- checkout in progress.> >> [start checkout 2008.12.22 06:30:10]> >> cd /tmp/build-cm3-20081222-063006-5jaOwe/build> >> cvs return value: 0> >> [end checkout 2008.12.22 06:49:53]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 06:49:55 -- compile in progress.> >> [start compile 2008.12.22 06:49:55]> >> compile return value: 0> >> [end compile 2008.12.22 06:55:09]> >> COMPILE_RETURN = 1> >> *** COMPILE FAILED> >> removing build tree /tmp/build-cm3-20081222-063006-5jaOwe ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> TESTHOSTNAME=niagara> >> WS=/homes/hosking/work/cm3-ws/niagara-2008-12-22-11-57-14> >> LASTREL=5.4.0> >> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0> >> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok> >> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok> >> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current> >> CM3_OSTYPE=POSIX> >> CM3_TARGET=SOLgnu> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> CM3CVSSERVER=birch.elegosoft.com> >> CM3CVSROOT=birch.elegosoft.com:/usr/cvs> >> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz> >> testing ssh birch.elegosoft.com..> >> ssh birch.elegosoft.com ok> >> Building cm3.> >> Tinderbox Tree: "cm3"> >> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"> >>> >> creating log file /tmp/build-cm3-20081222-065716-Oha4Uk/log.txt> >>> >> ---> >>> >> checkout, compile and test of cm3 ...> >> 2008.12.22 06:57:16 -- checkout in progress.> >> [start checkout 2008.12.22 06:57:18]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> cvs return value: 0> >> [end checkout 2008.12.22 07:16:39]> >> CHECKOUT_RETURN = 0> >> --> >> 2008.12.22 07:16:41 -- compile in progress.> >> [start compile 2008.12.22 07:16:41]> >> compile return value: 0> >> [end compile 2008.12.22 08:02:05]> >> COMPILE_RETURN = 0> >> 2008.12.22 08:02:22 -- tests in progress.> >> [start run-tests 2008.12.22 08:02:22]> >> cd /tmp/build-cm3-20081222-065716-Oha4Uk/build> >> [end run-tests 2008.12.22 08:02:22]> >> TESTS_RETURN = 0> >> 2008.12.22 08:02:22 -- checkout, compile and test run done.> >>> >> ---> >>> >> removing build tree /tmp/build-cm3-20081222-065716-Oha4Uk ...> >> cleaning CM3 workspaces...> >> /homes/hosking/work/cm3-ws/niagara-*> >>> >> cleaning regression test log files...> >> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*> >>> >> cleaning m3test log files...> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr> >>> >> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract> >>> >> cleaning snapshot files...> >> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz> >>> >> cleaning package reports...> >> /tmp/cm3-pkg-report-SOLgnu-*.html> >>> >> done.> > > > -- > Olaf Wagner -- elego Software Solutions GmbH> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney.bates at wichita.edu Tue Dec 23 18:28:36 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 11:28:36 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> Message-ID: <49511FC4.2050903@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 2^16-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. I think we already have a reasonably designed abstraction for TEXT, a bit of it built in to the language and the rest in Text.i3. Only problem 1) affects the abstraction. Addressing only 1) for now: It ought to cause minimal grief to just change WIDECHAR so it has a big enough value range for all the Unicode values, probably 32 bits in today's world. Surely nobody has written any code that assumes BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly hard change to adapt old code to, since the static rules of the language would point directly to most places that would need to be fixed. That leaves the wide TEXT literals. It is easy to forget, and I have to keep reminding myself, but (assuming again that nobody has gotten their fingers improperly into the implementation pie) there is currently no such thing as a "WIDETEXT" type. Both kinds of literals are of type TEXT. They are just different lexical rules for specifying literal values of type TEXT. A bit like '16' and '16_10' are different ways of writing the same value, with the same type INTEGER. This differs from the CHAR and WIDECHAR literals, which really are of different types. The one change needed to the W"..." literals would be to allow escape sequences inside for giving characters numerically. Right now, the \x0123 form of escape requires exactly 4 hex digits. If we added a new alternative escape letter, in addition to the 'x', that required, say, exactly 8 hex digits, then these literals could express characters in the needed extra space, without affecting existing code. I suppose, for consistency and completeness, we should also add a new octal escape sequence that was long enough for the full new range. And, we would also need to allow the same new escape sequences in WIDECHAR literals. This would solve the lack of sufficient range problem and have either minimal or zero impact on existing code. Compiler and library changes would not be trivial, but quite reasonable. I believe problems 2) and 3) can be addressed solely by working on the TEXT implementation - Rodney Bates From dragisha at m3w.org Tue Dec 23 19:43:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 19:43:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <49511FC4.2050903@wichita.edu> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> Message-ID: <1230057780.10385.23.camel@faramir.m3w.org> As my mother's tongue uses two alphabets for writing, Latin covered by ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph correspondence and three digraphs in Latin variant I think I have over-average experience with non-Latin1 alphabets, in various areas. If I have to express what we call widetext literal in my code, I will have to work with Unicode tables and pick character by character. Tedious! What I would do is - switch my keyboard to either Latin or Cyrillic mapping and - imagine that!!! - just type! Thus getting UTF-8 characters into my source. My example literal would be: CONST MyNameInCyrillic = "??????? ?????"; MyNameInLatin = "Dragi?a Duri?"; You can see or not these glyphs, depending on your MUA and to some degree on MTA's in transit. With all WIDE* talk it is what I am using. Me being example guy from non-Latin1 world. How many of you are non-Latin1 people and using 16bit "W literals" ? dd On Tue, 2008-12-23 at 11:28 -0600, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values that > exceed 2^16-1. > ... > I think we already have a reasonably designed abstraction for TEXT, > a bit of it built in to the language and the rest in Text.i3. Only > problem 1) affects the abstraction. Addressing only 1) for now: > > It ought to cause minimal grief to just change WIDECHAR so it has a > big enough value range for all the Unicode values, probably 32 bits > in today's world. Surely nobody has written any code that assumes > BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > hard change to adapt old code to, since the static rules of the > language would point directly to most places that would need to be > fixed. > > That leaves the wide TEXT literals. It is easy to forget, and I have > to keep reminding myself, but (assuming again that nobody has gotten > their fingers improperly into the implementation pie) there is > currently > no such thing as a "WIDETEXT" type. Both kinds of literals are of > type > TEXT. They are just different lexical rules for specifying literal > values of type TEXT. A bit like '16' and '16_10' are different ways > of writing the same value, with the same type INTEGER. This differs > from the CHAR and WIDECHAR literals, which really are of different > types. > > The one change needed to the W"..." literals would be to allow > escape sequences inside for giving characters numerically. Right > now, the \x0123 form of escape requires exactly 4 hex digits. If > we added a new alternative escape letter, in addition to the 'x', that > required, say, exactly 8 hex digits, then these literals could express > characters in the needed extra space, without affecting existing code. > I suppose, for consistency and completeness, we should also add a new > octal escape sequence that was long enough for the full new range. > > And, we would also need to allow the same new escape sequences in > WIDECHAR > literals. > ... > > - Rodney Bates > -- Dragi?a Duri? From hendrik at topoi.pooq.com Wed Dec 24 14:00:15 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:00:15 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <20081224130015.GA1001@topoi.pooq.com> On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. I see it. All we need to do to make this work is say that Modula-3 programs are encoded in Unicode. That and some work under the hood so make it work. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? Well, I'm from the Latin-1 world. I've had to debug a compiler that was mistreating Korean strings (this was before Unicode), and I've written a program that used UTF-8 hiragana. But I only visit the nonLatin-1 world. I don't live there. -- hendrik From hendrik at topoi.pooq.com Wed Dec 24 14:10:30 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 08:10:30 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224130015.GA1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> Message-ID: <20081224131030.GD1001@topoi.pooq.com> On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > CONST > > MyNameInCyrillic = "??????? ?????"; > > MyNameInLatin = "Dragi?a Duri?"; > > > > You can see or not these glyphs, depending on your MUA and to some > > degree on MTA's in transit. > > I see it. All we need to do to make this work is say that Modula-3 > programs are encoded in Unicode. That and some work under the hood so > make it work. Well, probably a *lot* of work. -- hendrik From dragisha at m3w.org Tue Dec 23 20:34:00 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Tue, 23 Dec 2008 20:34:00 +0100 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <20081224131030.GD1001@topoi.pooq.com> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> Message-ID: <1230060840.10385.30.camel@faramir.m3w.org> On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > CONST > > > MyNameInCyrillic = "??????? ?????"; > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > degree on MTA's in transit. > > > > I see it. All we need to do to make this work is say that Modula-3 > > programs are encoded in Unicode. That and some work under the hood so > > make it work. > > Well, probably a *lot* of work. Not at all. I am already using it. Just typing into. What does not work and I am not using it "length in glyphs". Not important for me, but not hard to implement. Either directly in Modula-3 or using external C lib. Search for UTF-8 pattern in UTF-8 text works - of course. Also - partitioning of TEXT's by *any real* criteria. By position of CHAR, or by position of any substring. > > -- hendrik > -- Dragi?a Duri? From mika at async.caltech.edu Tue Dec 23 22:15:39 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 13:15:39 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Tue, 23 Dec 2008 11:28:36 CST." <49511FC4.2050903@wichita.edu> Message-ID: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> "Rodney M. Bates" writes: ... >It ought to cause minimal grief to just change WIDECHAR so it has a >big enough value range for all the Unicode values, probably 32 bits >in today's world. Surely nobody has written any code that assumes >BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly ... Minor issue, I know, but... Pickles? Mika From rodney.bates at wichita.edu Tue Dec 23 23:31:54 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:31:54 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <495166DA.2090606@wichita.edu> Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? Yes, I have been thinking about this too, and some implementation work would be required here. It should not be too hard to use the existing mechanism that pickles/unpickles INTEGER fields of differing sizes on different architectures and do the same thing with different sizes of WIDECHAR. And apparently having had my fingers inside of pickles recently more than anybody else, I would expect to be the one to do it. It is likely that network objects containing such fields would want similar treatment. > > Mika From rodney.bates at wichita.edu Tue Dec 23 23:45:47 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 16:45:47 -0600 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230057780.10385.23.camel@faramir.m3w.org> References: <200812200719.mBK7J242062696@camembert.async.caltech.edu> <2405EF3C-1037-412F-8CC6-B615B4FCD6C3@cs.purdue.edu> <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> Message-ID: <49516A1B.8050903@wichita.edu> They look very believable on my email client. Perhaps I glossed over this. Yes, if your keyboard/editor can be set to handle the glyphs you want directly, then you can always use them in literals, of either kind, instead of escape sequences. I am sure we need more implementation work here, but I think the TEXT abstraction is fine in this respect. We still need escapes too, as an alternative for when using a keyboard/display setting isn't convenient, and it is for this purpose I am suggesting this extension to the TEXT literals, as part of making Unicode support complete. Dragi?a Duri? wrote: > As my mother's tongue uses two alphabets for writing, Latin covered by > ISO-8859-2 and Cyrillic covered by ISO-8859-5, with one-to-one glyph > correspondence and three digraphs in Latin variant I think I have > over-average experience with non-Latin1 alphabets, in various areas. > > If I have to express what we call widetext literal in my code, I will > have to work with Unicode tables and pick character by character. > Tedious! > > What I would do is - switch my keyboard to either Latin or Cyrillic > mapping and - imagine that!!! - just type! Thus getting UTF-8 characters > into my source. My example literal would be: > > CONST > MyNameInCyrillic = "??????? ?????"; > MyNameInLatin = "Dragi?a Duri?"; > > You can see or not these glyphs, depending on your MUA and to some > degree on MTA's in transit. > > With all WIDE* talk it is what I am using. Me being example guy from > non-Latin1 world. How many of you are non-Latin1 people and using 16bit > "W literals" ? > - Rodney Bates From rodney.bates at wichita.edu Wed Dec 24 00:22:35 2008 From: rodney.bates at wichita.edu (Rodney M. Bates) Date: Tue, 23 Dec 2008 17:22:35 -0600 Subject: [M3devel] This disgusting TEXT business Message-ID: <495172BB.9070906@wichita.edu> I hear three problems with CM3 TEXT: 1) WIDECHAR and the TEXT implementation won't handle Unicode values that exceed 216-1. 2) The CM3 TEXT implementation has serious inefficiencies in at least some realistic cases. 3) We want some kind of compatibility with other software. As for 2), the implementation could be greatly improved without changing the existing TEXT abstraction. It could even be improved a lot without changing the existing data structures, only the algorithms. Back in March, I conjectured about improving get_chars for a concatenation by recursing on only one side and iterating on the other, but I never did anything about it. Text.Cat could flatten concatenations that are short into one of the atomic representations. It could perhaps do a two-level rebalance of concatenation trees. All these would help at least somewhat with the performance consequences of building a string one character at a time by concatenation. And of course, ignoring the needs of those who have violated the TEXT abstraction, we could come up with entirely new internal representations. We could even just add some new ones and support mixtures of both the old and new ones. This would really just be an expansion of what we already have, which has several representations, with all the operations dynamically checking and adapting to them. On the other hand, I do know from hard experience, that the implementation size and complexity go up surprisingly as the number of alternative representations goes up. It's really a sort of cartesion product of operations, operands for each operation, and representations for each operand. Something that has not been mentioned is the _space_ overhead of the existing concatenation scheme. Proponents of the pure or nearly-pure OO language design philosophy (e.g. Smalltalk, Java) seem pretty oblivious to how much you lose with lots of separately-allocated heap objects connected by pointers. Aside from the extra pointers, there is allocator/GC overhead per object and fragmentation loss. Then there's loss of reference locality, leading to bigger working sets, which spills back into time overhead. And with heap-allocated open arrays, there are additional 1+NoOfDimensions words as well. So I am strongly in favor of cutting down on the number of separate objects, wherever reasonable. Rodney Bates From hendrik at topoi.pooq.com Wed Dec 24 19:00:01 2008 From: hendrik at topoi.pooq.com (hendrik at topoi.pooq.com) Date: Wed, 24 Dec 2008 13:00:01 -0500 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <1230060840.10385.30.camel@faramir.m3w.org> References: <1229764245.18256.148.camel@faramir.m3w.org> <20081220202755.yvow7v74sgg8c48w@mail.elegosoft.com> <20081221154015.GA6543@jack.stsp.name> <1C48CBCB-7ECD-4A6C-B4A2-6145DD651758@darko.org> <49511FC4.2050903@wichita.edu> <1230057780.10385.23.camel@faramir.m3w.org> <20081224130015.GA1001@topoi.pooq.com> <20081224131030.GD1001@topoi.pooq.com> <1230060840.10385.30.camel@faramir.m3w.org> Message-ID: <20081224180001.GA1449@topoi.pooq.com> On Tue, Dec 23, 2008 at 08:34:00PM +0100, Dragi?a Duri? wrote: > On Wed, 2008-12-24 at 08:10 -0500, hendrik at topoi.pooq.com wrote: > > On Wed, Dec 24, 2008 at 08:00:15AM -0500, hendrik at topoi.pooq.com wrote: > > > On Tue, Dec 23, 2008 at 07:43:00PM +0100, Dragi?a Duri? wrote: > > > > > > > > CONST > > > > MyNameInCyrillic = "??????? ?????"; > > > > MyNameInLatin = "Dragi?a Duri?"; > > > > > > > > You can see or not these glyphs, depending on your MUA and to some > > > > degree on MTA's in transit. > > > > > > I see it. All we need to do to make this work is say that Modula-3 > > > programs are encoded in Unicode. That and some work under the hood so > > > make it work. > > > > Well, probably a *lot* of work. > > Not at all. I am already using it. Just typing into. > > What does not work and I am not using it "length in glyphs". Not > important for me, but not hard to implement. Either directly in Modula-3 > or using external C lib. > > Search for UTF-8 pattern in UTF-8 text works - of course. Also - > partitioning of TEXT's by *any real* criteria. By position of CHAR, or > by position of any substring. I see. You are just using the existing TEXT to store the bytes of UTF-8. That seems one of the simplest ways of accomodating UTF-8, in fact. No conversions in or out, just work in a pure UTF-8 environment. The "work" I talked about would be the effert necessary to implement all the existing methods that use character counts as character counts. Treating them as using byte counts is indeed simple. What remains is to build operations that deal in these UTF-8 TEXTs as being strings of UTF-8 characters, with operations like fetching and storing characters instead of bytes. Perhaps what is really needed is clarity in the specification, so we know which operations to use on bytes, and which on characters, and abolish the myth that character counts are an efficient way of accessing characters. In retrospect, maybe there should have been a type BYTE, a type CHARACTER, and a type TEXT, all conceptually separate. TEXT could be accessed by byte or character operations; and indexes into TEXT would be byte offsets. This lacks some generality that might have been needed in the days of machines with word-addressable 29-bit words, but those days seem to be past. What seems to block introducing this approach seems to be the prevalence of code that uses CHARACTER as a synonym for BYTE. Do we, for some kind of compatibility, have to use CHARACTER when we mean BYTE, and something else when we want to say CHARACTER? In the long run, will be be more inconvenienced by having to rewrite code that uses CHARACTER to represent characters, or code that uses CHARACTER to represent bytes? It is already possible to use 0..255 to represent bytes, isn't it? Or am I wrong in this? Might this mean four-byte integers that happen to have a restricted range of values? -- hendrik From darko at darko.org Wed Dec 24 01:20:27 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:20:27 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <495172BB.9070906@wichita.edu> References: <495172BB.9070906@wichita.edu> Message-ID: <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> I agree with your list of problems, but you don't need a cartesian product of operations for string operations between multiple representations. All you need is iteration through the characters of a string and concatenation with another string for each representation. The object interface I proposed is little more than a wrapper for the existing Text interface plus the iteration method. The getData and setData methods in that object provide an interface to external APIs. On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > I hear three problems with CM3 TEXT: > > 1) WIDECHAR and the TEXT implementation won't handle Unicode values > that > exceed 216-1. > > 2) The CM3 TEXT implementation has serious inefficiencies in at > least some > realistic cases. > > 3) We want some kind of compatibility with other software. > > As for 2), the implementation could be greatly improved without > changing > the existing TEXT abstraction. It could even be improved a lot > without > changing the existing data structures, only the algorithms. > > Back in March, I conjectured about improving get_chars for a > concatenation > by recursing on only one side and iterating on the other, but I > never did > anything about it. Text.Cat could flatten concatenations that are > short > into one of the atomic representations. It could perhaps do a two- > level > rebalance of concatenation trees. All these would help at least > somewhat > with the performance consequences of building a string one character > at > a time by concatenation. > > And of course, ignoring the needs of those who have violated the TEXT > abstraction, we could come up with entirely new internal > representations. > We could even just add some new ones and support mixtures of both > the old > and new ones. This would really just be an expansion of what we > already > have, which has several representations, with all the operations > dynamically > checking and adapting to them. > > On the other hand, I do know from hard experience, that the > implementation > size and complexity go up surprisingly as the number of alternative > representations goes up. It's really a sort of cartesion product of > operations, operands for each operation, and representations for each > operand. > > Something that has not been mentioned is the _space_ overhead of the > existing concatenation scheme. Proponents of the pure or nearly-pure > OO language design philosophy (e.g. Smalltalk, Java) seem pretty > oblivious to how much you lose with lots of separately-allocated > heap objects connected by pointers. > > Aside from the extra pointers, there is allocator/GC overhead > per object and fragmentation loss. Then there's loss of > reference locality, leading to bigger working sets, which spills > back into time overhead. And with heap-allocated open arrays, > there are additional 1+NoOfDimensions words as well. So I am > strongly in favor of cutting down on the number of separate objects, > wherever reasonable. > > Rodney Bates From darko at darko.org Wed Dec 24 01:27:20 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:27:20 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> References: <200812232115.mBNLFdBB043926@camembert.async.caltech.edu> Message-ID: <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Besides breaking pickles (which isn't minor to those it affects) it also has the effect of doubling the size of String16 arrays and any wide strings or arrays. Why change the meaning of an existing type, possibly breaking unknown software when we can just define another type? It's not like another built-in type will cause anyone problems. On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > "Rodney M. Bates" writes: > ... >> It ought to cause minimal grief to just change WIDECHAR so it has a >> big enough value range for all the Unicode values, probably 32 bits >> in today's world. Surely nobody has written any code that assumes >> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly > ... > > Minor issue, I know, but... Pickles? > > Mika From mika at async.caltech.edu Wed Dec 24 01:29:15 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:29:15 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:27:20 +0900." <8011A80A-EF16-40A4-A855-3A1AD5E9E084@darko.org> Message-ID: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Darko writes: >Besides breaking pickles (which isn't minor to those it affects) it >also has the effect of doubling the size of String16 arrays and any >wide strings or arrays. Why change the meaning of an existing type, >possibly breaking unknown software when we can just define another >type? It's not like another built-in type will cause anyone problems. stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: > >> "Rodney M. Bates" writes: >> ... >>> It ought to cause minimal grief to just change WIDECHAR so it has a >>> big enough value range for all the Unicode values, probably 32 bits >>> in today's world. Surely nobody has written any code that assumes >>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >> ... >> >> Minor issue, I know, but... Pickles? >> >> Mika From mika at async.caltech.edu Wed Dec 24 01:31:50 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:31:50 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:20:27 +0900." <470393D9-ABA3-48B9-8E78-FD57DCBE0902@darko.org> Message-ID: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Might I suggest an interface somewhat like the Rd/Wr interfaces? One could provide a buffer into which characters are read en masse instead of going through the method calls for each character.. Of course for equality this isn't necessarily more efficient since you often can do an early-out: although I believe that Text.Equal (when performance matters) usually does return TRUE.... I think it is important that basic TEXT operations be very, very fast. Many people care about that. Many of those that don't wouldn't be using a compiled language anyhow. Mika Darko writes: >I agree with your list of problems, but you don't need a cartesian >product of operations for string operations between multiple >representations. All you need is iteration through the characters of a >string and concatenation with another string for each representation. >The object interface I proposed is little more than a wrapper for the >existing Text interface plus the iteration method. The getData and >setData methods in that object provide an interface to external APIs. > > >On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: > >> I hear three problems with CM3 TEXT: >> >> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >> that >> exceed 216-1. >> >> 2) The CM3 TEXT implementation has serious inefficiencies in at >> least some >> realistic cases. >> >> 3) We want some kind of compatibility with other software. >> >> As for 2), the implementation could be greatly improved without >> changing >> the existing TEXT abstraction. It could even be improved a lot >> without >> changing the existing data structures, only the algorithms. >> >> Back in March, I conjectured about improving get_chars for a >> concatenation >> by recursing on only one side and iterating on the other, but I >> never did >> anything about it. Text.Cat could flatten concatenations that are >> short >> into one of the atomic representations. It could perhaps do a two- >> level >> rebalance of concatenation trees. All these would help at least >> somewhat >> with the performance consequences of building a string one character >> at >> a time by concatenation. >> >> And of course, ignoring the needs of those who have violated the TEXT >> abstraction, we could come up with entirely new internal >> representations. >> We could even just add some new ones and support mixtures of both >> the old >> and new ones. This would really just be an expansion of what we >> already >> have, which has several representations, with all the operations >> dynamically >> checking and adapting to them. >> >> On the other hand, I do know from hard experience, that the >> implementation >> size and complexity go up surprisingly as the number of alternative >> representations goes up. It's really a sort of cartesion product of >> operations, operands for each operation, and representations for each >> operand. >> >> Something that has not been mentioned is the _space_ overhead of the >> existing concatenation scheme. Proponents of the pure or nearly-pure >> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >> oblivious to how much you lose with lots of separately-allocated >> heap objects connected by pointers. >> >> Aside from the extra pointers, there is allocator/GC overhead >> per object and fragmentation loss. Then there's loss of >> reference locality, leading to bigger working sets, which spills >> back into time overhead. And with heap-allocated open arrays, >> there are additional 1+NoOfDimensions words as well. So I am >> strongly in favor of cutting down on the number of separate objects, >> wherever reasonable. >> >> Rodney Bates From darko at darko.org Wed Dec 24 01:35:37 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:35:37 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> References: <200812240029.mBO0TFnD050221@camembert.async.caltech.edu> Message-ID: <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Those don't involve data stored in files, can be fixed with a recompile and are only an issue if you reveal the object implementation of TEXT. On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > Darko writes: >> Besides breaking pickles (which isn't minor to those it affects) it >> also has the effect of doubling the size of String16 arrays and any >> wide strings or arrays. Why change the meaning of an existing type, >> possibly breaking unknown software when we can just define another >> type? It's not like another built-in type will cause anyone problems. > > stubgen, stablegen, ... :-) no free lunch here, I'm afraid. > >> >> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >> >>> "Rodney M. Bates" writes: >>> ... >>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>> big enough value range for all the Unicode values, probably 32 bits >>>> in today's world. Surely nobody has written any code that assumes >>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>> ... >>> >>> Minor issue, I know, but... Pickles? >>> >>> Mika From mika at async.caltech.edu Wed Dec 24 01:41:20 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 23 Dec 2008 16:41:20 -0800 Subject: [M3devel] This disgusting TEXT business In-Reply-To: Your message of "Wed, 24 Dec 2008 09:35:37 +0900." <866D18A8-2DA8-473A-94C4-33119325FDC1@darko.org> Message-ID: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Sorry, I think I misinterpreted what you meant by "built-in type". Darko writes: >Those don't involve data stored in files, can be fixed with a >recompile and are only an issue if you reveal the object >implementation of TEXT. > >On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: > >> Darko writes: >>> Besides breaking pickles (which isn't minor to those it affects) it >>> also has the effect of doubling the size of String16 arrays and any >>> wide strings or arrays. Why change the meaning of an existing type, >>> possibly breaking unknown software when we can just define another >>> type? It's not like another built-in type will cause anyone problems. >> >> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >> >>> >>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>> >>>> "Rodney M. Bates" writes: >>>> ... >>>>> It ought to cause minimal grief to just change WIDECHAR so it has a >>>>> big enough value range for all the Unicode values, probably 32 bits >>>>> in today's world. Surely nobody has written any code that assumes >>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a terribly >>>> ... >>>> >>>> Minor issue, I know, but... Pickles? >>>> >>>> Mika From darko at darko.org Wed Dec 24 01:47:36 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:47:36 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> References: <200812240031.mBO0Vois050341@camembert.async.caltech.edu> Message-ID: <064090A2-A17E-4CBF-9B7B-3A5B575C5706@darko.org> The proposed interface is like Rd/Wr in that it has a procedural interface on an opaque type that is revealed to be implemented as an object for each type of Rd/Wr. But the idea is to avoid the "one true" representation (which what the buffer is) because efficiency means different things to different people. If you do a lot of Unicode text and an UTF-32 representation is probably fastest. If you want to save space or you deal mostly with ASCII then UTF-8 is probably best. Going through each character is needed if you don't know the representation of the string you are dealing with. If you do know the format of the string you can do other operations. Since most applications will use one representation you will mostly be doing operations between strings of the same representations, which will be very fast, for instance you might call a local OS API to do the work. Having application defined representations means that you can adapt the string representation and operations to what you need to give you the performance you're after, or to choose one from an existing library that works best. On 24/12/2008, at 9:31 AM, Mika Nystrom wrote: > Might I suggest an interface somewhat like the Rd/Wr interfaces? > One could provide a buffer into which characters are read en masse > instead of going through the method calls for each character.. > > Of course for equality this isn't necessarily more efficient since > you often can do an early-out: although I believe that Text.Equal > (when performance matters) usually does return TRUE.... > > I think it is important that basic TEXT operations be very, very > fast. Many people care about that. Many of those that don't > wouldn't be using a compiled language anyhow. > > Mika > > Darko writes: >> I agree with your list of problems, but you don't need a cartesian >> product of operations for string operations between multiple >> representations. All you need is iteration through the characters >> of a >> string and concatenation with another string for each representation. >> The object interface I proposed is little more than a wrapper for the >> existing Text interface plus the iteration method. The getData and >> setData methods in that object provide an interface to external APIs. >> >> >> On 24/12/2008, at 8:22 AM, Rodney M. Bates wrote: >> >>> I hear three problems with CM3 TEXT: >>> >>> 1) WIDECHAR and the TEXT implementation won't handle Unicode values >>> that >>> exceed 216-1. >>> >>> 2) The CM3 TEXT implementation has serious inefficiencies in at >>> least some >>> realistic cases. >>> >>> 3) We want some kind of compatibility with other software. >>> >>> As for 2), the implementation could be greatly improved without >>> changing >>> the existing TEXT abstraction. It could even be improved a lot >>> without >>> changing the existing data structures, only the algorithms. >>> >>> Back in March, I conjectured about improving get_chars for a >>> concatenation >>> by recursing on only one side and iterating on the other, but I >>> never did >>> anything about it. Text.Cat could flatten concatenations that are >>> short >>> into one of the atomic representations. It could perhaps do a two- >>> level >>> rebalance of concatenation trees. All these would help at least >>> somewhat >>> with the performance consequences of building a string one character >>> at >>> a time by concatenation. >>> >>> And of course, ignoring the needs of those who have violated the >>> TEXT >>> abstraction, we could come up with entirely new internal >>> representations. >>> We could even just add some new ones and support mixtures of both >>> the old >>> and new ones. This would really just be an expansion of what we >>> already >>> have, which has several representations, with all the operations >>> dynamically >>> checking and adapting to them. >>> >>> On the other hand, I do know from hard experience, that the >>> implementation >>> size and complexity go up surprisingly as the number of alternative >>> representations goes up. It's really a sort of cartesion product of >>> operations, operands for each operation, and representations for >>> each >>> operand. >>> >>> Something that has not been mentioned is the _space_ overhead of the >>> existing concatenation scheme. Proponents of the pure or nearly- >>> pure >>> OO language design philosophy (e.g. Smalltalk, Java) seem pretty >>> oblivious to how much you lose with lots of separately-allocated >>> heap objects connected by pointers. >>> >>> Aside from the extra pointers, there is allocator/GC overhead >>> per object and fragmentation loss. Then there's loss of >>> reference locality, leading to bigger working sets, which spills >>> back into time overhead. And with heap-allocated open arrays, >>> there are additional 1+NoOfDimensions words as well. So I am >>> strongly in favor of cutting down on the number of separate objects, >>> wherever reasonable. >>> >>> Rodney Bates From darko at darko.org Wed Dec 24 01:50:53 2008 From: darko at darko.org (Darko) Date: Wed, 24 Dec 2008 09:50:53 +0900 Subject: [M3devel] This disgusting TEXT business In-Reply-To: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> References: <200812240041.mBO0fKS1050680@camembert.async.caltech.edu> Message-ID: Sorry I thought you were referring to TEXT representations. We could define UNICHAR = INTEGER, an alias, which would avoid those changes. That might be required since the Unicode character set seems to change constantly and it might not work to define it as an enumeration. On 24/12/2008, at 9:41 AM, Mika Nystrom wrote: > > Sorry, I think I misinterpreted what you meant by "built-in type". > > Darko writes: >> Those don't involve data stored in files, can be fixed with a >> recompile and are only an issue if you reveal the object >> implementation of TEXT. >> >> On 24/12/2008, at 9:29 AM, Mika Nystrom wrote: >> >>> Darko writes: >>>> Besides breaking pickles (which isn't minor to those it affects) it >>>> also has the effect of doubling the size of String16 arrays and any >>>> wide strings or arrays. Why change the meaning of an existing type, >>>> possibly breaking unknown software when we can just define another >>>> type? It's not like another built-in type will cause anyone >>>> problems. >>> >>> stubgen, stablegen, ... :-) no free lunch here, I'm afraid. >>> >>>> >>>> On 24/12/2008, at 6:15 AM, Mika Nystrom wrote: >>>> >>>>> "Rodney M. Bates" writes: >>>>> ... >>>>>> It ought to cause minimal grief to just change WIDECHAR so it >>>>>> has a >>>>>> big enough value range for all the Unicode values, probably 32 >>>>>> bits >>>>>> in today's world. Surely nobody has written any code that >>>>>> assumes >>>>>> BITSIZE(WIDECHAR)=16. ;-) Even if so, this shouldn't be a >>>>>> terribly >>>>> ... >>>>> >>>>> Minor issue, I know, but... Pickles? >>>>> >>>>> Mika From jay.krell at cornell.edu Wed Dec 24 09:00:39 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 24 Dec 2008 08:00:39 +0000 Subject: [M3devel] a text proposal?? Message-ID: NOTE I am not a vehement advocate of the below. I'm at least "floating" a "thought balloon". Like, doesn't this sound fairly ok and reasonable? I propose the following "one" "parameterized" representation: a block of memory with a stored allocated size, probably in bytes, but could be divided by character size (i.e. "length in chars" vs. "length in bytes") a stored character size, in bytes, which shall be 1, 2, or 4 a stored length, probably in bytes, but could be divided by character size a stored maximum character value, maybe This is too expensive to maintain in general. It could be "advisory". However, it is also somewhat redundant with the character size. There shall be one implementation, though part of it shall be in a generic module, parameterized by character size. (yeah, yeah, parameterized by whatever Modula-3 allows) Appending a string with character size n with a string with character size m shall result in a string with character size max(n, m). String append shall always result in a "flat array". Never linked lists. Strings are never "encoded". Characters shall only ever be zero extended. Maybe there shall be functions to create strings from encoded strings. e.g. from utf8 and whatever is the name for 16 bit unicode that contain 20 bit surrogate pairs or whatever they are called. And to encode strings. Concatenating shall grow the allocated size by at least a constant factor greater than one. Possibly 2, possibly smaller. If that fails, fallback to allocating only what is needed. [This probably makes no sense, given text immutability and that concat produces new texts. But it might make sense if "text" is a value type.] Concatenation of n texts shall be a primitive, not just 2 texts. [It looks like this is already the case.] I think TEXTs are considered immutable. Over-allocating for later in-place concatenation doesn't, I believe, contract that. However perhaps I misspoke, and the allocated size is stored with the pointer, and the text contains a pointer to that, and a length. And maybe an offset. But "internal pointers" are ok and therefore an offset not needed, right? Cop out rationale: Anyone who needs O(1) concatenation of very long strings, is probably using the wrong type. Since texts are immutable, it may be viable, or even cheap, to allow a text to stick around with multiple representations, possibly using "weak refs". That is, string shall have probably a function or functions to get at the raw chars. Either the user has to specify a char size he wants, and get back null if the wrong size, or perhaps if user asks for larger than is needed, the text realizes that representation on-demand? And then it sticks around, in case another client soon thereafter asks for the same wider-than-necessary representation? The first order implementation can be simpler though. It can either return null for all but the current representation, or realize and return, but not hold onto, whatever representation is requested. Note that I'm not familiar with the code (yet) so this might be partly or total nonsense. I'm particular uncertain about the ability to double allocation size as individual chars are appended to a text. Issues of normalization remain punted. They already exist. Functions could be provided for convert to composed, decomposed, etc. - Jay From dragisha at m3w.org Thu Dec 25 01:00:05 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 25 Dec 2008 01:00:05 +0100 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 Message-ID: <1230163205.7131.1.camel@faramir.m3w.org> Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode (via UTF-8 as internal rep) then we must also extend CHAR so it can represent any single Unicode glyph ? in fact CHAR becomes 32bit value instead of current 8bit. If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we are on UNICHAR route ? as proposed by Darko (IIRC). But ? down that road we have variations in very traditional interface Text.i3 ? Text.GetChar, Text.SetChars, Text.FromChar, Text.FromChars must have these two variants ? somehow. I see no elegant way to handle that if we insist on BITSIZE(CHAR) = 8. UNICHAR route also contains other branches, most of them analog to current Text8/Text16 mess. UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte buffers, is one problem. Not too hard to spot and fix, though. Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that way. If similar need happens in future, ie some runtime level data communication, I think we can do it at ?connection? level. Some marshalling would most probably always take place ? so why not add TEXT I/O to list of tasks needed? -- Dragi?a Duri? From dabenavidesd at yahoo.es Thu Dec 25 01:12:07 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 25 Dec 2008 00:12:07 +0000 (GMT) Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Message-ID: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Dear all, actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the? helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix INTERFACE). In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. You can read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess)? http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on? back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32..m3.html ). Happy holidays and hope this helps a bit more. --- El lun, 22/12/08, Mika Nystrom escribi?: De: Mika Nystrom Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c Para: "Jay" CC: mika at camembert.async.caltech.edu Fecha: lunes, 22 diciembre, 2008 10:49 I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now
-- changing my "=
>ino_t" to
"m3_ino_t".
I'm still
checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 -
 Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinbishop at bellsouth.net Thu Dec 25 03:14:31 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Wed, 24 Dec 2008 20:14:31 -0600 Subject: [M3devel] Modula-3 subreddit Message-ID: <4952EC87.5050709@bellsouth.net> Just wanted to say that I made a Modula-3 subreddit on Reddit (a while back) if anyone is interested in joining (and hopefully submitting interesting links :P) http://www.reddit.com/r/modula3/ From mika at async.caltech.edu Fri Dec 26 09:56:24 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 26 Dec 2008 00:56:24 -0800 Subject: [M3devel] (draft draft proposal) Unicode TEXT via BITSIZE(CHAR) = 32 In-Reply-To: Your message of "Thu, 25 Dec 2008 01:00:05 +0100." <1230163205.7131.1.camel@faramir.m3w.org> Message-ID: <200812260856.mBQ8uOZ0065080@camembert.async.caltech.edu> I think you may find "SET OF CHAR" hidden all over the place. Also a lot of code may depend on the fact that CHAR and C's "char" types match on a given platform. This is your UNSAFE objection, but I think it goes further than just buffers. Mika =?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >Basic building element of TEXT is CHAR. So, if we extend TEXT to Unicode >(via UTF-8 as internal rep) then we must also extend CHAR so it can >represent any single Unicode glyph ??? in fact CHAR becomes 32bit value >instead of current 8bit. > >If we insist to preserve BITSIZE(CHAR) = 8 (and I don't see why) then we >are on UNICHAR route ??? as proposed by Darko (IIRC). But ??? down that road >we have variations in very traditional interface Text.i3 ??? Text.GetChar, >Text.SetChars, Text.FromChar, Text.FromChars must have these two >variants ??? somehow. I see no elegant way to handle that if we insist on >BITSIZE(CHAR) = 8. > >UNICHAR route also contains other branches, most of them analog to >current Text8/Text16 mess. > >UNSAFE code written with various ARRAY OF CHAR which are, in fact, byte >buffers, is one problem. Not too hard to spot and fix, though. > >Current TEXT/WIDETEXT was implemented because CMASS JVM needed it that >way. If similar need happens in future, ie some runtime level data >communication, I think we can do it at ???connection??? level. Some >marshalling would most probably always take place ??? so why not add TEXT >I/O to list of tasks needed? > >-- >Dragi??a Duri?? From martinbishop at bellsouth.net Sat Dec 27 03:39:08 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Fri, 26 Dec 2008 20:39:08 -0600 Subject: [M3devel] Who runs modula3.org? Message-ID: <4955954C.5050102@bellsouth.net> Who runs modula3.org, or at least has access? Quite a few of the links are dead. Some link to m3.org, others link to gatekeeper (which is gone). Most have alternative pages hosted elsewhere, but almost all are archived at archive.org, if needed. From dragisha at m3w.org Sat Dec 27 08:02:11 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 27 Dec 2008 08:02:11 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <4955954C.5050102@bellsouth.net> References: <4955954C.5050102@bellsouth.net> Message-ID: <1230361331.7131.6.camel@faramir.m3w.org> Public reply, so I can be corrected by someone knowing better. WWW is AFAIK run by Elego, but content is version controlled from CVS, so IMO it can be changed usual way - by editing and commiting. I don't know about permissions and update procedures for this, but you can browse src repo a bit and see for yourself. On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: > Who runs modula3.org, or at least has access? Quite a few of the links > are dead. Some link to m3.org, others link to gatekeeper (which is > gone). Most have alternative pages hosted elsewhere, but almost all are > archived at archive.org, if needed. -- Dragi?a Duri? From wagner at elegosoft.com Sat Dec 27 15:43:49 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 27 Dec 2008 15:43:49 +0100 Subject: [M3devel] Who runs modula3.org? In-Reply-To: <1230361331.7131.6.camel@faramir.m3w.org> References: <4955954C.5050102@bellsouth.net> <1230361331.7131.6.camel@faramir.m3w.org> Message-ID: <20081227154349.5lyajia0w0k88wko@mail.elegosoft.com> Quoting Dragi?a Duri? : > Public reply, so I can be corrected by someone knowing better. > > WWW is AFAIK run by Elego, but content is version controlled from CVS, > so IMO it can be changed usual way - by editing and commiting. > > I don't know about permissions and update procedures for this, but you > can browse src repo a bit and see for yourself. > > On Fri, 2008-12-26 at 20:39 -0600, Martin Bishop wrote: >> Who runs modula3.org, or at least has access? Quite a few of the links >> are dead. Some link to m3.org, others link to gatekeeper (which is >> gone). Most have alternative pages hosted elsewhere, but almost all are >> archived at archive.org, if needed. Elego took over the hosting of modula3.org from Igencorp (formerly Critical Mass) run by Farshad Nayeri one or two years ago, I think. The sources are checked in somewhere on birch.elego.de (though I don't know the exact path offhand (I'm not at home right now)) and can be edited and shipped by anyone with ssh access to birch. You can also send me diffs and I'll try to apply them, which may take some days of course. Any improvements and corrections are very welcome! Olaf > -- > Dragi?a Duri? > > -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Sat Dec 27 20:48:46 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 19:48:46 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 Message-ID: fyi: I've been looking at Utime.i3. One of the variations is whether or not struct_tm contains gmtoff and zone at the end. C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 are wrong on some systems. They unconditionally add the two fields, and read from memory that doesn't necessarily have them. It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). I'll double check Solaris and Irix soon or next week. I'll try to come up with a fix shortly. It will surely involve deleting those two files. And breaking any users of Utime.localtime and Utime.gmtime. - Jay From jay.krell at cornell.edu Sat Dec 27 21:12:50 2008 From: jay.krell at cornell.edu (Jay) Date: Sat, 27 Dec 2008 20:12:50 +0000 Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 In-Reply-To: References: Message-ID: Are localtime_r and gmtime_r really not widely available these days? I don't have a Darwin system handy this week. That is, we have DateBsd.m3, DateLinux.m3, DatePosix.m3. Excerpt: readonly _DateImpls = { "AMD64_DARWIN": "DateBsd", "AMD64_LINUX": "DateBsd", "FreeBSD*" : "DateBsd", "LINUXLIBC6" : "DateBsd", "MIPS64_OPENBSD" : "DateBsd", "NT386" : "DateLinux", "NT386GNU" : "DateLinux", "SOLgnu" : "DatePosix", "SOLsun" : "DatePosix", "SPARC" : "DateBsd", "*_LINUX" : "DateBsd", "SPARC64_SOLARIS" : "DatePosix", "*_DARWIN" : "DateBsd", "*_OPENBSD" : "DateBsd", } DateLinux lacks reentreent functions, and lacks gmtoff/zone in struct tm. It is only used by Cygwin. And Linux 1.x which I assume doesn't matter. This appears to be an oversight by me, the reentrant functions exist. DateBsd lacks reentrant functions, and has gmtoff/zone in struct tm. It is very widely used. DatePosix has reentrant functions, and lacks gmtoff/zone in struct tm. I rather suspect that everyone can just use DatePosix. ? Or at least that DateBsd can use reentrant functions. Eliminating unnecessary locks/globals always seems good. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Sat, 27 Dec 2008 19:48:46 +0000 > Subject: [M3devel] fyi Utime.i3 / UtimeExtra.i3 > > > fyi: > > > I've been looking at Utime.i3. > One of the variations is whether or not struct_tm contains gmtoff and zone at the end. > > > C:\dev2\cm3.2\m3-mail\postcard\src\UtimeExtra.i3 > C:\dev2\cm3.2\m3-mail\webcard\src\UtimeExtra.i3 > > > are wrong on some systems. > They unconditionally add the two fields, and read from memory that doesn't necessarily have them. > > > It is ok on FreeBSD, OpenBSD, NetBSD, modern Linux (maybe not Linux 1.x), Darwin. > It appears wrong on Cygwin, Solaris, HP-UX, Irix, OS/2 (did OS/2 ever work?). > I'll double check Solaris and Irix soon or next week. > > > I'll try to come up with a fix shortly. > It will surely involve deleting those two files. > And breaking any users of Utime.localtime and Utime.gmtime. > > > - Jay From mika at async.caltech.edu Tue Dec 23 04:49:25 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 22 Dec 2008 19:49:25 -0800 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: Your message of "Sun, 21 Dec 2008 17:57:00 GMT." Message-ID: <200812230349.mBN3nPem008337@camembert.async.caltech.edu> I think Daniel has a point. He said "UNSAFE", not "unsafe". "UNSAFE" means something very specific in Modula-3, and operations that are UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and hence only can be used in UNSAFE MODULEs. If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that is a problem that really shouldn't be allowed to get worse. This is all covered in sections 1.4.5 and 2.1 of the Green Book. I'm not sure exactly what you guys are talking about but I would appreciate it if I could rely on the Green Book for programming in Modula-3.... Not everything that's <*EXTERNAL*> is UNSAFE, by the way. It is perfectly possible for a routine coded in C or Fortran to export a non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can definitely export an UNSAFE interface. I think if you interpret section 2.1 a bit loosely, but I think still within the spirit of the authors' intent, you can go as far as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE interfaces) as long as the program needs to use at least one UNSAFE INTERFACE in order to trigger the error. The contrapositive of this is that it should not be possible for a Modula-3 program that imports only non-UNSAFE interfaces to cause an unchecked runtime error. Ever. Some examples...., in the following, assume that CrashHorribly can cause an unchecked runtime error when called. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. This is OK since an importer has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. INTERFACE A; TYPE T = BRANDED ROOT; END A. UNSAFE INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Might be OK, as long as the only way to get an unchecked runtime error in C is by passing in a B.T. If you can get an unchecked runtime error in C.CrashHorribly without importing B (say by passing in a bare A.T or a "home-made" subtype of A), it's wrong. INTERFACE A; TYPE T = BRANDED ROOT; END A. INTERFACE B; IMPORT A; TYPE T <: A.T; END B. INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. Definitely wrong. Mika Jay writes: >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >The real world is unsafe.Safe areas are created by unsafe code=2C which mus= >t be=2C to a certain extent=2C bug free=2C in order to provide as safe as a= >dvertised. I don't believe what I added is any more untrusted/unsafe than = >what was there before.Well=2C maybe in a minor way. m3core has "lots" of "= >external" functions=2C which are called by "lots" of functions at higher le= >vels..ok=2C maybe not "lots"=2C but if you look at the m3core/src/unix dire= >ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-= >common -- this is a fairly minimal set of stuff that is being used by preex= >isting code. A LITTLE bit I added beyond what was there. If you look at all= > the other directories=2C you'll find lots more external stuff=2C most of w= >hich is not used within the CM3 codebase (but could be used by folks outsid= >e). It is true I may have introduced external where it wasn't previously= >=2C but I don't think it matters.It is prevalent one level down. Moving som= >e of it one level up is ok I believe. =20 >Non-Posix does potentially have a place in .c files.It depends.Basically=2C= > as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes = >hard to read=2C and at some point=2C you split the files into separate file= >s=2C "no" #ifdefs -- or conceptually each file can be viewed as wrapped in = >an #ifdef or other platform detection.You need #ifdef in any C that uses 64= > bit integers=2C for example. Current Microsoft compilers do support "long = >long" but __int64 supporg goes way back=2C and __int64 is a better name any= >way. >What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong". >=20 >=20 >The names "char"=2C "short"=2C "int"=2C "long"=2C "long long" are mostly ju= >st funny mistaken names. >The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized types= > such as "size_t" and "diff_t". >Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section. >=20 >Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C. >=20 > - Jay > > > >Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: m3dev= >el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit]= > FilePosixC.c > > > >Dear all:I'm glad that Jay and all the people around are making contributio= >ns=2C so we can claim that as some day there were a good number of supporte= >d platforms nowadays we also have.However I have a question=2C but its more= > a suggestion=2C the modification Tony is referring is not as standard as w= >e would like since the type safety is affected with the change introduced a= >llowing <*EXTERNAL*> (untrusted) functions in the INTERFACE FilePosix=2C it= >'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C so = >we may want to mark as UNSAFE INTERFACE=2C however doing that I think some = >clients could not claim any more they are safe so that "fix" is not feasib= >le.Can you push to a lower level (as in the UNSAFE FilePosix implementation= > module before committing the change) or make it available in other Interfa= >ce already available=2C why do we need that change anyway? Also IMHO we sh= >ould leave out what is not POSIX (we should not care for WIN64 although I k= >now is needed by the C compiler preprocessor) out of that file.Although I d= >on't have the formal specification or program=2C but I can remember libm3 c= >ode was specified and mechanically checked with the Extended Static Checker= > Modula-3 (which is not sound also=2C that is could be cheated=2C see http:= >//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 )=2C = > that would be a good reason for keeping as safe as possible and close to h= >istorical interfaces and just rework implementations (even if they are not = >safe) as needed. I know that implementation is not safe but at least having= > it well enough implemented the ESC/M3 checker and the compiler would tell = >us the library should work Ok and is safe and (hopefully) without the error= >s detected by the ESC as we care for the details in the UNSAFE implementati= >on.There are any special interfaces we should have as "standard" in the low= >er level runtime (hard enough question I know)=2C can we give some hints/id= >eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay ll.edu> escribi=F3: >De: Jay Asunto: Re: [M3devel] serial/CygwinPara: "m3= >devel" =2C "Tony" Fecha: vier= >nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ it is o= >k now -- changing my "ino_t" to"m3_ino_t".I'm still checking though.And I o= >nly checked SOLgnu. I will check SOLsun. - Jay = > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >The real world is unsafe.
Safe areas are created by unsafe code=2C which= > must be=2C to a certain extent=2C bug free=2C in order to provide as safe = >as advertised.
 =3B
 =3B
I don't believe what I added is a= >ny more untrusted/unsafe than what was there before.
Well=2C maybe in a = >minor way.
 =3B
 =3B
m3core has "lots" of "external" funct= >ions=2C which are called by "lots" of functions at higher levels..ok=2C may= >be not "lots"=2C but if you look at the m3core/src/unix directories that I = >have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this = >is a fairly minimal set of stuff that is being used by preexisting code. A = >LITTLE bit I added beyond what was there. If you look at all the other dire= >ctories=2C you'll find lots more external stuff=2C most of which is not use= >d within the CM3 codebase (but could be used by folks outside).
 =3B= >
 =3B
It is true I may have introduced external where it wasn't p= >reviously=2C but I don't think it matters.
It is prevalent one level dow= >n. Moving some of it one level up is ok I believe. =3B
 =3B
>
Non-Posix does potentially have a place in .c files.
It depends.
= >Basically=2C as I see things=2C a small amount of #ifdef is ok.
A lot of= > #ifdef becomes hard to read=2C and at some point=2C you split the files in= >to separate files=2C "no" #ifdefs -- or conceptually each file can be viewe= >d as wrapped in an #ifdef or other platform detection.
You need #ifdef i= >n any =3BC that uses 64 bit integers=2C for example. Current Microsoft&= >nbsp=3Bcompilers do support "long long" but __int64 supporg goes way back= >=2C and __int64 is a better name anyway.
>What is the 128 bit integer type to be called -- "__int128" or "long long l= >ong".
> =3B
> =3B
>The names "char"=2C "short"=2C "int"=2C "long"=2C =3B"long long" = >=3Bare mostly just funny mistaken names.
>The right names to provide are explicitly sized=2C int8=2C =3Buint8=2C = >int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized = >types such as "size_t" and "diff_t".
>Modula-3 I think is reasonable here=2C since it's primary integral type is = >pointer sized=2C and it has "bits x for ..."=2C though the other types prob= >ably ought to be built in and blessed somewhere=2C not just in the Csupport= > library section.
>
 =3B
>Not sure what "Posix" means=2C since it might incorporate by-reference ANSI= > C.
> =3B
> =3B
 =3B- Jay



> >
>
>
Date: Sat=2C 20 Dec 2008 00:09:41 +0000
From: dabenavidesd at yahoo.es<= >BR>To: m3devel at elegosoft.com
Subject: [M3devel] Rv: Re: serial/Cygwin wa= >s Re: [M3commit] FilePosixC.c



> > > >
Dear all:
I'm glad that Jay and all the people around a= >re making contributions=2C so we can claim that as some day there were a go= >od number of supported platforms nowadays we also have.
However I have a= > question=2C but its more a suggestion=2C the modification Tony is referrin= >g is not as standard as we would like since the type safety is affected wit= >h the change introduced allowing <=3B*>EXTERNAL*>=3B (untrusted) functions = >in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 (IMHO= > it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C = >however doing that I think some clients could not =3B claim any more th= >ey are safe so that "fix" is not feasible.
Can you push to a lower level= > (as in the UNSAFE FilePosix implementation module before committing the ch= >ange) or make it available in other Interface already available=2C why do w= >e need that change anyway? =3B Also IMHO we should leave out what is no= >t POSIX (we should not care for WIN64 although I know is needed by the C co= >mpiler preprocessor) out of that file.
Although I don't have the formal = >specification or program=2C but I can remember libm3 code was specified and= > mechanically checked with the Extended Static Checker Modula-3 (which is n= >ot sound also=2C that is could be cheated=2C see http://www.researchchannel= >.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B that wo= >uld be a good reason for keeping as safe as possible and close to historica= >l interfaces and just rework implementations (even if they are not safe) as= > needed. I know that implementation is not safe but at least having it well= > enough implemented the ESC/M3 checker and the compiler would tell us the l= >ibrary should work Ok and is safe and (hopefully) without the errors detect= >ed by the ESC as we care for the details in the UNSAFE implementation.
T= >here are any special interfaces we should have as "standard" in the lower l= >evel runtime (hard enough question I know)=2C can we give some hints/ideas = >on that?
Thanks in advance



--- El vie=2C 19/12/08=2C J= >ay <=3Bjay.krell at cornell.edu>=3B escribi=F3:
>
De: Jay <=3Bj= >ay.krell at cornell.edu>=3B
Asunto: Re: [M3devel] serial/Cygwin
Para: = >"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C "Tony" <=3Bhosking at cs.purd= >ue.edu>=3B
Fecha: viernes=2C 19 diciembre=2C 2008 1:25

>=
>=3B And I'll fix SOLsun/gnu.

I /think/ it is ok now -- changing my "=
>ino_t" to
"m3_ino_t".
I'm still checking though.
And I only checke=
>d SOLgnu. I will check SOLsun.
 
 - Jay

TD>

>= > >--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- From jay.krell at cornell.edu Sun Dec 28 01:19:24 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:19:24 +0000 Subject: [M3devel] PPC_LINUX Message-ID: I think PPC_LINUX has been broken for a few years, and not released since 5.2.6. It should be ok now and current 5.2.7 min and std distributions are at http://modula3.elegosoft.com/cm3/uploaded-archives We might want to fix user threads? - Jay From jay.krell at cornell.edu Sun Dec 28 01:22:20 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 00:22:20 +0000 Subject: [M3devel] port names? Message-ID: Again..what do people want for port names? MSDOS? Nice and short in the old style. I386_MSDOS? I think so. DJGPP? Nice and short in the old style. I386_DJGPP? Good because it identifies more -- the compiler, linker, C runtime. As opposed to say "C_WATCOM_DOS" or "I386_DOS_WATCOM" or such. DOS? Again, nice and short in the old style. I386_SOLARIS? Presumbly. X86_SOLARIS? AMD32_SOLARIS? :) IA32_SOLARIS? - Jay From jay.krell at cornell.edu Sun Dec 28 12:00:11 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 11:00:11 +0000 Subject: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c In-Reply-To: <892265.61271.qm@web28603.mail.ukl.yahoo.com> References: <892265.61271.qm@web28603.mail.ukl.yahoo.com> Message-ID: another take on this.. Cloning C headers in Modula-3 is both error prone and unsafe. It is not subject to static checking, is not verifiably correct, and errors lead to the same sort of type system violations that unsafe code can cause. I have found a few bugs here and there in the cloned headers, mostly wrt alignment. I replaced it with something probably still unsafe, but far less error prone. - Jay ________________________________ > Date: Thu, 25 Dec 2008 00:12:07 +0000 > From: dabenavidesd at yahoo.es > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > > Dear all, > actually I have found on The Modula-3 book of Harbison that it is expected to be exported safe INTERFACEs from safe and also UNSAFE MODULES, but the programmer should take care of the tournarounds of some failing ("not type safe checked") operations in the later case that could end crashing the entire systems without checked runtime error. > I agree that some important needs of this procedures are highly dependent on the platform/system specific helper procedures (please take a look of the old fashioned m3tohtml web page of http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/POSIX/FilePosix.m3.html of how the Unix operating system dependent UNSAFE INTERFACEs are isolated from the helper Procedures of Unix/POSIX platform for FilePosix UNSAFE MODULE and then become accesable through a safe FilePosix > INTERFACE). > In fact this is supposed to disallow unchecked runtime errors as much as is possible to happen (maybe just at the most > operating system and specific level that is on m3core and not ever in libm3 which I think may be more secure than just type safe checked as I mentioned). > Definitely we need platform low level handling, say POSIX, WIN32 (possibly new WIN64 nowadays) specific code, but to do that is good rule of the thumb have the unsafe code like in those days of a huge number of supported operating system code (for UNIX and later for WIN32 platforms) that tells the system history. > Actually the Modula-3 compiler wouldn't be able to tell anything about C and if I'm not misunderstanding nor misinterpreting this mechanism of Modula-3 UNSAFE INTERFACE-UNSAFE MODULE -safe INTERFACE if I can make such a name was used to avoid unchecked runtime errors happened at pure Modula-3 parts of the system (such as libm3), and maybe a little more beyond that, having defined platform specific and operating system low level interfaces which can be more easily reworked. > You can > read about this issue a bit more on the Partial Revelations article about safety issue http://modula3.elegosoft.com/pm3/pkg/modula3/src/discussion/partialRev.html . > Apart of that, I don't know if you would like the kind of documentation is reported on the SRC M3 generated file more than the corresponding one in nowadays (not produced every day I guess) http://opencm3.net/doc/help/gen_html/libm3/src/os/POSIX/FilePosix.m3.html I think could be useful to get back the same style of documentation procedures and correspondent files on this platform specific files (might be helpful to know where are the correspondant Interfaces on WIN platform as It wasn't on back days http://www.cs.tut.fi/lintula/manual/modula3/m3sources/html/os/src/WIN32/FileWin32.m3.html ). > > Happy holidays and hope this helps a bit more. > > > --- El lun, 22/12/08, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] Rv: Re: serial/Cygwin was Re: [M3commit] FilePosixC.c > Para: "Jay" > CC: mika at camembert.async.caltech.edu > Fecha: lunes, 22 diciembre, 2008 > 10:49 > > > I think Daniel has a point. He said "UNSAFE", not > "unsafe". "UNSAFE" > means something very specific in Modula-3, and operations that are > UNSAFE are only permitted to be exported by UNSAFE INTERFACEs and > hence only can be used in UNSAFE MODULEs. > > If, however, there are UNSAFE interfaces in non-UNSAFE INTERFACEs, that > is a problem that really shouldn't be allowed to get worse. > > This is all covered in sections 1.4.5 and 2.1 of the Green Book. > I'm not sure exactly what you guys are talking about but I would > appreciate it if I could rely on the Green Book for programming in > Modula-3.... > > Not everything that's is UNSAFE, by the way. It is > perfectly possible for a routine coded in C or Fortran to export a > non-UNSAFE Modula-3 interface. Conversely, a Modula-3 MODULE can > definitely export an UNSAFE interface. > > I think if you interpret section 2.1 a bit loosely, > but > I think > still within the spirit of the authors' intent, you can go as far > as to say that it's OK to introduce UNSAFE code (even in non-UNSAFE > interfaces) as long as the program needs to use at least one UNSAFE > INTERFACE in order to trigger the error. The contrapositive of > this is that it should not be possible for a Modula-3 program that > imports only non-UNSAFE interfaces to cause an unchecked runtime > error. Ever. > > Some examples...., in the following, assume that CrashHorribly > can cause an unchecked runtime error when called. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > This is OK since an importer has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T UNSAFE INTERFACE C; IMPORT > A; > PROCEDURE CrashHorribly(arg : A.T); END C. > > Still OK, an importer still has to be UNSAFE to trigger CrashHorribly. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > UNSAFE INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C. > > Might be OK, as long as the only way to get an unchecked runtime > error in C is by passing in a B.T. If you can get an unchecked > runtime error in C.CrashHorribly without importing B (say by passing > in a bare A.T or a "home-made" subtype of A), it's wrong. > > INTERFACE A; TYPE T = BRANDED ROOT; END A. > INTERFACE B; IMPORT A; TYPE T INTERFACE C; IMPORT A; PROCEDURE CrashHorribly(arg : A.T); END C.. > > Definitely wrong. > > Mika > > > Jay writes: >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/plain; charset="iso-8859-1" >>Content-Transfer-Encoding: > > quoted-printable >> >> >>The real world is unsafe.Safe areas are created by unsafe code=2C which > mus= >>t be=2C to a certain extent=2C bug free=2C in order to provide as safe as > a= >>dvertised. I don't believe what I added is any more untrusted/unsafe > than = >>what was there before.Well=2C maybe in a minor way. m3core has > "lots" of "= >>external" functions=2C which are called by "lots" of > functions at higher le= >>vels..ok=2C maybe not "lots"=2C but if you look at the > m3core/src/unix dire= >>ctories that I have worked on -- cygwin=2C openbsd=2C amd64_linux=2C > linux-= >>common -- this is a fairly minimal set of stuff that is being used by > preex= >>isting code. A LITTLE bit I added beyond what was there. If you look at > all= >> the other directories=2C you'll find lots more external stuff=2C most > of w= >>hich is not used within the CM3 codebase (but could > be > used by folks > outsid= >>e). It is true I may have introduced external where it wasn't > previously= >>=2C but I don't think it matters.It is prevalent one level down. Moving > som= >>e of it one level up is ok I believe. =20 >>Non-Posix does potentially have a place in .c files.It > depends.Basically=2C= >> as I see things=2C a small amount of #ifdef is ok.A lot of #ifdef becomes > = >>hard to read=2C and at some point=2C you split the files into separate > file= >>s=2C "no" #ifdefs -- or conceptually each file can be viewed as > wrapped in = >>an #ifdef or other platform detection.You need #ifdef in any C that uses > 64= >> bit integers=2C for example. Current Microsoft compilers do support > "long = >>long" but __int64 supporg goes way back=2C and __int64 is a better > name any= >>way. >>What is the 128 bit integer type to be called -- "__int128" or > "long long > > l= >>ong". >>=20 >>=20 >>The names "char"=2C "short"=2C "int"=2C > "long"=2C "long long" are mostly ju= >>st funny mistaken names. >>The right names to provide are explicitly sized=2C int8=2C uint8=2C int16= >>=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > types= >> such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >>=20 >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >>=20 >> - Jay >> >> >> >>Date: Sat=2C 20 Dec 2008 00:09:41 +0000From: dabenavidesd at yahoo.esTo: > m3dev= >>el at elegosoft.comSubject: [M3devel] Rv: Re: serial/Cygwin > was > Re: > [M3commit]= >> FilePosixC.c >> >> >> >>Dear all:I'm glad that Jay and all the people around are making > contributio= >>ns=2C so we can claim that as some day there were a good number of > supporte= >>d platforms nowadays we also have.However I have a question=2C but its > more= >> a suggestion=2C the modification Tony is referring is not as standard as > w= >>e would like since the type safety is affected with the change introduced > a= >>llowing (untrusted) functions in the INTERFACE > FilePosix=2C it= >>'s an implementation issue of the CM3 (IMHO it should warn) compiler=2C > so = >>we may want to mark as UNSAFE INTERFACE=2C however doing that I think some > = >>clients could not claim any more they are safe so that "fix" is > not feasib= >>le.Can you push to a lower level (as in the UNSAFE FilePosix > implementation= >> module > before > committing the change) or make it available in other > Interfa= >>ce already available=2C why do we need that change anyway? Also IMHO we > sh= >>ould leave out what is not POSIX (we should not care for WIN64 although I > k= >>now is needed by the C compiler preprocessor) out of that file.Although I > d= >>on't have the formal specification or program=2C but I can remember > libm3 c= >>ode was specified and mechanically checked with the Extended Static > Checker= >> Modula-3 (which is not sound also=2C that is could be cheated=2C see > http:= >>//www.researchchannel.org/prog/displayevent.aspx?rID=3D2761&fID=3D345 > )=2C = >> that would be a good reason for keeping as safe as possible and close to > h= >>istorical interfaces and just rework implementations (even if they are not > = >>safe) as needed. I know that implementation is not safe but at least > having= >> it well > enough > implemented the ESC/M3 checker and the compiler would tell > = >>us the library should work Ok and is safe and (hopefully) without the > error= >>s detected by the ESC as we care for the details in the UNSAFE > implementati= >>on.There are any special interfaces we should have as "standard" > in the low= >>er level runtime (hard enough question I know)=2C can we give some > hints/id= >>eas on that?Thanks in advance--- El vie=2C 19/12/08=2C Jay >>>ll.edu> escribi=F3: >>De: Jay Asunto: Re: [M3devel] > serial/CygwinPara: "m3= >>devel" =2C "Tony" > Fecha: vier= >>nes=2C 19 diciembre=2C 2008 1:25> And I'll fix SOLsun/gnu.I /think/ > it is o= >>k now -- changing my "ino_t" to"m3_ino_t".I'm still > checking though.And I o= >>nly checked SOLgnu. I will check SOLsun. - > Jay > = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_ >>Content-Type: text/html; charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>The real world is unsafe. Safe areas are created by unsafe code=2C > which= >> must be=2C to a certain extent=2C bug free=2C in order to provide as safe > = >>as advertised. =3B =3B I > don't believe what I added is a= >>ny more untrusted/unsafe than what was there before. Well=2C maybe > in a = >>minor way. =3B =3B m3core > has > "lots" of > "external" funct= >>ions=2C which are called by "lots" of functions at higher > levels..ok=2C may= >>be not "lots"=2C but if you look at the m3core/src/unix > directories that I = >>have worked on -- cygwin=2C openbsd=2C amd64_linux=2C linux-common -- this > = >>is a fairly minimal set of stuff that is being used by preexisting code. A > = >>LITTLE bit I added beyond what was there. If you look at all the other > dire= >>ctories=2C you'll find lots more external stuff=2C most of which is not > use= >>d within the CM3 codebase (but could be used by folks > outside). =3B= >> =3B It is true I may have introduced external > where it wasn't p= >>reviously=2C but I don't think it matters. It is prevalent one > level dow= >>n. Moving some of it one level up is ok > I > believe. =3B =3B >> Non-Posix > does potentially have a place in .c files. It > depends. = >>Basically=2C as I see things=2C a small amount of #ifdef is ok. A > lot of= >> #ifdef becomes hard to read=2C and at some point=2C you split the files > in= >>to separate files=2C "no" #ifdefs -- or conceptually each file > can be viewe= >>d as wrapped in an #ifdef or other platform detection. You need > #ifdef i= >>n any =3BC that uses 64 bit integers=2C for example. Current > Microsoft&= >>nbsp=3Bcompilers do support "long long" but __int64 supporg goes > way back= >>=2C and __int64 is a better name anyway. >>What is the 128 bit integer type to be called -- "__int128" or > "long long l= >>ong". >> =3B >> =3B >>The names "char"=2C > "short"=2C "int"=2C > "long"=2C =3B"long long" = >>=3Bare mostly just > funny mistaken names. >>The right names to provide are explicitly sized=2C > int8=2C =3Buint8=2C = >>int16=2C uint16=2C int32=2C uint32=2C int64=2C uint64=2C and pointer sized > = >>types such as "size_t" and "diff_t". >>Modula-3 I think is reasonable here=2C since it's primary integral type > is = >>pointer sized=2C and it has "bits x for ..."=2C though the other > types prob= >>ably ought to be built in and blessed somewhere=2C not just in the > Csupport= >> library section. >> =3B >>Not sure what "Posix" means=2C since it might incorporate > by-reference ANSI= >> C. >> =3B >> =3B =3B- > Jay >> >> ________________________________ > id=3DstopSpelling> >> >> Date: Sat=2C 20 Dec 2008 00:09:41 > +0000 From: > dabenavidesd at yahoo.es<= >>BR>To: m3devel at elegosoft.com Subject: [M3devel] Rv: Re: > serial/Cygwin wa= >>s Re: [M3commit] FilePosixC.c >> >> >> >> Dear all: I'm glad that Jay and all the > people around a= >>re making contributions=2C so we can claim that as some day there were a > go= >>od number of supported platforms nowadays we also have. However I > have a= >> question=2C but its more a suggestion=2C the modification Tony is > referrin= >>g is not as standard as we would like since the type safety is affected > wit= >>h the change introduced allowing > <=3B> class=3DEC_current-rev>*>>>EXTERNAL> class=3DEC_current-rev>*>=3B > (untrusted) functions = >>in the INTERFACE FilePosix=2C it's an implementation issue of the CM3 > (IMHO= >> it should warn) compiler=2C so we may want to mark as UNSAFE INTERFACE=2C > = >>however doing that I think some clients could not =3B claim any > more th= >>ey are safe so that "fix" is not feasible. Can you push > to a lower level= >> (as in the UNSAFE FilePosix implementation module before committing the > ch= >>ange) or make it available in other Interface already available=2C why do > w= >>e need that change anyway? =3B Also IMHO we should leave out what > is no= >>t POSIX (we should not care for WIN64 although I know is needed by the C > co= >>mpiler preprocessor) out of that file. Although I don't have > the > formal = >>specification or program=2C but I can remember libm3 code was > specified > and= >> mechanically checked with the Extended Static Checker Modula-3 (which is > n= >>ot sound also=2C that is could be cheated=2C see > http://www.researchchannel= >>.org/prog/displayevent.aspx?rID=3D2761&=3BfID=3D345 )=2C =3B > that wo= >>uld be a good reason for keeping as safe as possible and close to > historica= >>l interfaces and just rework implementations (even if they are not safe) > as= >> needed. I know that implementation is not safe but at least having it > well= >> enough implemented the ESC/M3 checker and the compiler would tell us the > l= >>ibrary should work Ok and is safe and (hopefully) without the errors > detect= >>ed by the ESC as we care for the details in the UNSAFE > implementation. T= >>here are any special interfaces we should have as "standard" > in > the lower l= >>evel runtime (hard enough question I know)=2C can we give > some hints/ideas > = >>on that? Thanks in > advance --- El vie=2C > 19/12/08=2C J= >>ay <=3Bjay.krell at cornell.edu>=3B > escribi=F3: >> > 5px">De: Jay <=3Bj= >>ay.krell at cornell.edu>=3B Asunto: Re: [M3devel] > serial/Cygwin Para: = >>"m3devel" <=3Bm3devel at elegosoft.com>=3B=2C > "Tony" <=3Bhosking at cs.purd= >>ue.edu>=3B Fecha: viernes=2C 19 diciembre=2C 2008 > 1:25 >= >>=3B And I'll fix SOLsun/gnu. I /think/ it is ok now > -- changing my "= >>ino_t" to "m3_ino_t". I'm still > checking though. And I only > checke= >>d SOLgnu. I will check SOLsun. - > Jay > >>TD> >> = >> >>--_4c4a182e-6ca9-4b3c-a937-0fea11bffd83_-- > > From martinbishop at bellsouth.net Sun Dec 28 23:14:14 2008 From: martinbishop at bellsouth.net (Martin Bishop) Date: Sun, 28 Dec 2008 16:14:14 -0600 Subject: [M3devel] Arbitrary precision integers Message-ID: <4957FA36.8080308@bellsouth.net> Does Modula-3 (or one of it's implementations) support arbitrary precision integers? From dabenavidesd at yahoo.es Sun Dec 28 23:48:35 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Dec 2008 22:48:35 +0000 (GMT) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> Message-ID: <460649.13613.qm@web28612.mail.ukl.yahoo.com> Hi, I guess BigInteger is a big integer of arbitrary size, you can look on http://www.opencm3.net/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html Also there is another effort made in a? B Sc's Thesis: http://www.emis.de/journals/NSJOM/Papers/30_3/NSJOM_30_3_123_130.pdf may be helpful, I haven't contact them, it should be helpful to ask. ? --- El dom, 28/12/08, Martin Bishop wrote: De: Martin Bishop Asunto: [M3devel] Arbitrary precision integers Para: m3devel at elegosoft.com Fecha: domingo, 28 diciembre, 2008 5:14 Does Modula-3 (or one of it's implementations) support arbitrary precision integers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sun Dec 28 23:50:37 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:50:37 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081228100334.C41F3B04048@birch.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> Message-ID: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> This change makes me uneasy -- do you mean to say we have a uniform definition of sem_t when the different targets have different definitions? :-O On 28 Dec 2008, at 11:03, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/28 11:03:34 > > Modified files: > cm3/m3-libs/m3core/src/unix/Common/: m3makefile > cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile > cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile > Added files: > cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 > Removed files: > cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 > cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 > > Log message: > another common header file -- however this bought by defining sem_t > to have the maximum size and alignment of any system; > Solaris's 48 byte 64bit aligned version > The CM3 code base has one static sem_t in the pthread code. From hosking at cs.purdue.edu Sun Dec 28 23:46:44 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 09:46:44 +1100 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> The language itself does not support such things, though there may be a library implementing them somewhere. On 29 Dec 2008, at 09:14, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? From jay.krell at cornell.edu Mon Dec 29 00:49:25 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:49:25 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. There is only one sem_t in the system. It is global. This change just makes it usually larger and more aligned than necessary. The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. Usysdep.i3 seems best? - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jkrell at elego.de > Date: Mon, 29 Dec 2008 09:50:37 +1100 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > This change makes me uneasy -- do you mean to say we have a uniform > definition of sem_t when the different targets have different > definitions? :-O > > On 28 Dec 2008, at 11:03, Jay Krell wrote: > >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/28 11:03:34 >> >> Modified files: >> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >> Added files: >> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >> Removed files: >> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >> >> Log message: >> another common header file -- however this bought by defining sem_t >> to have the maximum size and alignment of any system; >> Solaris's 48 byte 64bit aligned version >> The CM3 code base has one static sem_t in the pthread code. > From jay.krell at cornell.edu Mon Dec 29 00:59:58 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:59:58 +0000 Subject: [M3devel] Arbitrary precision integers In-Reply-To: <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> References: <4957FA36.8080308@bellsouth.net> <008E462F-0C23-41EF-B741-15AEEB2772DE@cs.purdue.edu> Message-ID: > though there may be > a library implementing them somewhere. The compiler itself has three such libraries embedded within it. :) Depending on how you define things. The front end cm3 does at least 64 bit integer math via 16 bit integers. cm3cg uses gmp and mpfr. The goals are probably the same for all three -- evaluate constant expressions correctly for the target. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: martinbishop at bellsouth.net > Date: Mon, 29 Dec 2008 09:46:44 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Arbitrary precision integers > > The language itself does not support such things, though there may be > a library implementing them somewhere. > > On 29 Dec 2008, at 09:14, Martin Bishop wrote: > >> Does Modula-3 (or one of it's implementations) support arbitrary >> precision integers? > From jay.krell at cornell.edu Mon Dec 29 00:55:16 2008 From: jay.krell at cornell.edu (Jay) Date: Sun, 28 Dec 2008 23:55:16 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Note that I've been avoiding touching "mainstream" or already existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. Perhaps I should go ahead and cross that line, as my testability/testing allows? Slightly linked question is if user threads should remain. They don't work, I believe on any platform, but I understand could easily be made to. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; jkrell at elego.de > Date: Sun, 28 Dec 2008 23:49:25 +0000 > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3commit] CVS Update: cm3 > > > This one I can undo and still feel ok -- just push sem_t into Utypes.i3 and expose it from a common Usem.i3. > There is only one sem_t in the system. > It is global. This change just makes it usually larger and more aligned than necessary. > > > The danger would be if sem_t is ever embedded in something else OS-defined, or if the OS ever returned a pointer to one that Modula-3 code copied. > > > In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all system-dependent stuff. > Even existant Utypes.i3 have much redundancy -- all the ulong/ushort/int8/int16/int32/int64 types are always the same, just stuff like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t should always be INT64 or UINT64. > > > Usysdep.i3 seems best? > > > - Jay > > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jkrell at elego.de >> Date: Mon, 29 Dec 2008 09:50:37 +1100 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> This change makes me uneasy -- do you mean to say we have a uniform >> definition of sem_t when the different targets have different >> definitions? :-O >> >> On 28 Dec 2008, at 11:03, Jay Krell wrote: >> >>> CVSROOT: /usr/cvs >>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>> >>> Modified files: >>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>> Added files: >>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>> Removed files: >>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>> >>> Log message: >>> another common header file -- however this bought by defining sem_t >>> to have the maximum size and alignment of any system; >>> Solaris's 48 byte 64bit aligned version >>> The CM3 code base has one static sem_t in the pthread code. >> From hosking at cs.purdue.edu Mon Dec 29 01:14:02 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 11:14:02 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: User threads used to work on Solaris (since it uses getcontext/ setcontext to do thread switching instead of setjmp/longjmp) and these should work similarly on Linux/BSD/OSX. The only problem is that we need to use makecontext on some platforms to get a proper basis context for new threads (as opposed to using the hackier code that is there in ThreadPosix.m3. On 29 Dec 2008, at 10:55, Jay wrote: > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my testability/ > testing allows? > > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; jkrell at elego.de >> Date: Sun, 28 Dec 2008 23:49:25 +0000 >> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >> Subject: Re: [M3commit] CVS Update: cm3 >> >> >> This one I can undo and still feel ok -- just push sem_t into >> Utypes.i3 and expose it from a common Usem.i3. >> There is only one sem_t in the system. >> It is global. This change just makes it usually larger and more >> aligned than necessary. >> >> >> The danger would be if sem_t is ever embedded in something else OS- >> defined, or if the OS ever returned a pointer to one that Modula-3 >> code copied. >> >> >> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >> system-dependent stuff. >> Even existant Utypes.i3 have much redundancy -- all the ulong/ >> ushort/int8/int16/int32/int64 types are always the same, just stuff >> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >> should always be INT64 or UINT64. >> >> >> Usysdep.i3 seems best? >> >> >> - Jay >> >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jkrell at elego.de >>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> This change makes me uneasy -- do you mean to say we have a uniform >>> definition of sem_t when the different targets have different >>> definitions? :-O >>> >>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>> >>>> CVSROOT: /usr/cvs >>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>> >>>> Modified files: >>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>> Added files: >>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>> Removed files: >>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>> >>>> Log message: >>>> another common header file -- however this bought by defining sem_t >>>> to have the maximum size and alignment of any system; >>>> Solaris's 48 byte 64bit aligned version >>>> The CM3 code base has one static sem_t in the pthread code. >>> From jay.krell at cornell.edu Mon Dec 29 01:21:17 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 00:21:17 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: Understood. You (someone) also has to fix the initialization. I think when you (you) moved the various user thread globals (?), such as the handler chain, into one nice thread object, "switchable" via just one pointer, it broke. I didn't look at the change in depth, I'm making a leap in my description of what you did. It is a /very/ reasonable design (or even likely The Correct Design), to collect up all the globals that represent a "thread" into one heap allocated struct...except it is an "object". I don't have SOLgnu/sun handy, this week, but can try it next week. I expect it fails as PPC_LINUX was. I THINK the "object" needs to be changed to a "traced ref" and then it can be initialized using, maybe, a preallocated global, for the first thread, or at least using regular "Malloc" and not going through the garbage collection...but I'm really just not sure. If I get X86_MSDOS/DJGPP up and going, I'll be a bit more motivated..but I think I'll so I386_SOLARIS first, since it is perhaps more modern/mainstream. You know, I tried having it NEW the object on-demand, but I think that ended up recursing to about the same place -- PushEFrame. It's "difficult" to repro now for me, this week, based on hardware/OS availability,because I ripped out PPC_LINUX user thread support (unless I install NetBSD/FreeBSD in a VM or use birch, all of which are, honestly, easy). - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 11:14:02 +1100 > CC: jkrell at elego.de; m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > User threads used to work on Solaris (since it uses getcontext/ > setcontext to do thread switching instead of setjmp/longjmp) and these > should work similarly on Linux/BSD/OSX. The only problem is that we > need to use makecontext on some platforms to get a proper basis > context for new threads (as opposed to using the hackier code that is > there in ThreadPosix.m3. > > On 29 Dec 2008, at 10:55, Jay wrote: > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? >> >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. >> >> - Jay >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: hosking at cs.purdue.edu; jkrell at elego.de >>> Date: Sun, 28 Dec 2008 23:49:25 +0000 >>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>> Subject: Re: [M3commit] CVS Update: cm3 >>> >>> >>> This one I can undo and still feel ok -- just push sem_t into >>> Utypes.i3 and expose it from a common Usem.i3. >>> There is only one sem_t in the system. >>> It is global. This change just makes it usually larger and more >>> aligned than necessary. >>> >>> >>> The danger would be if sem_t is ever embedded in something else OS- >>> defined, or if the OS ever returned a pointer to one that Modula-3 >>> code copied. >>> >>> >>> In general, I'm thinking either Utypes.i3 or some new Uinternal.i3 >>> or Usysdep.i3 or Utypes.i3 + Uconstants.i3 should be where most/all >>> system-dependent stuff. >>> Even existant Utypes.i3 have much redundancy -- all the ulong/ >>> ushort/int8/int16/int32/int64 types are always the same, just stuff >>> like nlink_t, mode_t, gid_t, uid_t, off_t vary. Though imho off_t >>> should always be INT64 or UINT64. >>> >>> >>> Usysdep.i3 seems best? >>> >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> From: hosking at cs.purdue.edu >>>> To: jkrell at elego.de >>>> Date: Mon, 29 Dec 2008 09:50:37 +1100 >>>> CC: m3devel at elegosoft.com; m3commit at elegosoft.com >>>> Subject: Re: [M3commit] CVS Update: cm3 >>>> >>>> This change makes me uneasy -- do you mean to say we have a uniform >>>> definition of sem_t when the different targets have different >>>> definitions? :-O >>>> >>>> On 28 Dec 2008, at 11:03, Jay Krell wrote: >>>> >>>>> CVSROOT: /usr/cvs >>>>> Changes by: jkrell at birch. 08/12/28 11:03:34 >>>>> >>>>> Modified files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: m3makefile >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: m3makefile >>>>> Added files: >>>>> cm3/m3-libs/m3core/src/unix/Common/: Usem.i3 >>>>> Removed files: >>>>> cm3/m3-libs/m3core/src/unix/linux-common/: Usem.i3 >>>>> cm3/m3-libs/m3core/src/unix/openbsd-common/: Usem.i3 >>>>> >>>>> Log message: >>>>> another common header file -- however this bought by defining sem_t >>>>> to have the maximum size and alignment of any system; >>>>> Solaris's 48 byte 64bit aligned version >>>>> The CM3 code base has one static sem_t in the pthread code. >>>> > From lemming at henning-thielemann.de Mon Dec 29 01:42:24 2008 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 29 Dec 2008 01:42:24 +0100 (CET) Subject: [M3devel] Arbitrary precision integers In-Reply-To: <4957FA36.8080308@bellsouth.net> References: <4957FA36.8080308@bellsouth.net> Message-ID: On Sun, 28 Dec 2008, Martin Bishop wrote: > Does Modula-3 (or one of it's implementations) support arbitrary > precision integers? http://modula3.elegosoft.com/cm3/doc/help/gen_html/arithmetic/src/basictypes/biginteger/BigInteger.i3.html It's not well optimized and maybe not enough tested. If you find it too slow, you might write wrapper to the GNU multiprecision library GMP. Maybe SWIG can help writing the wrapper. (or you try Haskell, which also uses GMP :-) From wagner at elegosoft.com Mon Dec 29 08:46:55 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 29 Dec 2008 08:46:55 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> Message-ID: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Quoting Jay : > > Note that I've been avoiding touching "mainstream" or already > existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. > > Perhaps I should go ahead and cross that line, as my > testability/testing allows? I wanted to comment on this before. I've no objection at trying to remove duplicated code as far as possible, but when changing system interfaces (Uxxx.is), please make sure that they really work on the target platform in question; if possible, add some tests. > Slightly linked question is if user threads should remain. > They don't work, I believe on any platform, but I understand could > easily be made to. I was not aware that user threads are broken; why? Anyway, this code should not be removed; I was rather hoping for a compile-time option to switch between user and system threads in cm3. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hosking at cs.purdue.edu Mon Dec 29 09:16:13 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 29 Dec 2008 19:16:13 +1100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> Message-ID: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > Quoting Jay : > >> >> Note that I've been avoiding touching "mainstream" or already >> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >> >> Perhaps I should go ahead and cross that line, as my testability/ >> testing allows? > > I wanted to comment on this before. I've no objection at trying to > remove duplicated code as far as possible, but when changing system > interfaces (Uxxx.is), please make sure that they really work on the > target platform in question; if possible, add some tests. > >> Slightly linked question is if user threads should remain. >> They don't work, I believe on any platform, but I understand could >> easily be made to. > > I was not aware that user threads are broken; why? > Anyway, this code should not be removed; I was rather hoping for a > compile-time option to switch between user and system threads in cm3. Definitely doable! I've been meaning to do this for a long time now. > > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, > Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 > 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: > Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > From jay.krell at cornell.edu Mon Dec 29 09:47:02 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 08:47:02 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> References: <20081228100334.C41F3B04048@birch.elegosoft.com> <3AD2B7A9-A060-4E03-B8BA-08B2D6028BB4@cs.purdue.edu> <20081229084655.smjz3zvzqww0ko88@mail.elegosoft.com> <40BCD3AA-2019-4601-B669-2E21EBC08C55@cs.purdue.edu> Message-ID: cm3 -DNOPTHREAD compiles m3core for user threads. I /sort of/ (almost) haven't removed anything. I did remove PPC_LINUX stuff. In new platforms (openbsd, {amd64,sparc}_linux, cygwin), I might have initially added partial support for user threads, never complete, tested, working support, and then removed it. User threads are broken because PushEFrame uses uninitialized NULL self. PushEFrame is always called before ThreadF__Init (which initializes self), but the other two threading libraries (which don't have "self" at all) initialize on demand what they need in PushEFrame (the thread local handler stack). I think self/Thread.T needs to be a traced ref, instead of an object, so the initial value can be to a preallocated static/global. I'm not sure of the language here. "object" seems to deprive you of the "value type". Whereas refs do not. I tried allocating self on demand in PushEFrame but that didn't work. It ended up back in PushEFrame. I tried more tactics after that, but then gave up. Or move the handle stack out of Thread.T and into a separate global like it was. Either change would not affect pthreads or NT threads. I would like to see how much I can whittle down m3core of platform-specific stuff, and then maybe see about adding user thread support for the platforms that never had it. I'll add I386_SOLARIS soon, and don't want to duplicate a "large" directory of files as part of that. I really dislike the multiplicity of such a system. I have another chunk of m3core about to go though, for systems that don't have user threads. Maybe not one that is worthwhile. I'm still not much of a believer in user threads. Essentially I have always had kernel threads, with a processor-portable interface, and one machine-wide scheduler..seems good.. Every thread mechanism needs its own locking mechanism, which won't interace with any other thread or locking mechanism. Kind of like picking a C compiler or library -- you want there to be just one obvious choice per-platform and have everyone use it, so that there is interop and better overall scheduling. You know..Modula-3 I think has problems when the executable doesn't use the Modula-3 runtime and/or when threads are created outside the runtime, at then run Modula-3 code. That seems unfortunate. I /should/ be able to write a "plugin .dll" in Modula-3 and have it loaded by a C or C++ application, and have garbage collection work. Win32 is also kind of wonky here, in that it has "fibers". Fibers are like swapcontext, plus the stack allocation is handled for you. Anyone using them has to implement their own scheduler and their own locking mechanisms. Win32 critical sections, mutexes, etc. don't interact "correctly" with them. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: wagner at elegosoft.com > Date: Mon, 29 Dec 2008 19:16:13 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On 29 Dec 2008, at 18:46, Olaf Wagner wrote: > >> Quoting Jay : >> >>> >>> Note that I've been avoiding touching "mainstream" or already >>> existant platforms -- FreeBSD, LINUXLIBC6, *_DARWIN. >>> >>> Perhaps I should go ahead and cross that line, as my testability/ >>> testing allows? >> >> I wanted to comment on this before. I've no objection at trying to >> remove duplicated code as far as possible, but when changing system >> interfaces (Uxxx.is), please make sure that they really work on the >> target platform in question; if possible, add some tests. >> >>> Slightly linked question is if user threads should remain. >>> They don't work, I believe on any platform, but I understand could >>> easily be made to. >> >> I was not aware that user threads are broken; why? >> Anyway, this code should not be removed; I was rather hoping for a >> compile-time option to switch between user and system threads in cm3. > > Definitely doable! I've been meaning to do this for a long time now. > >> >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, >> Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 >> 45 86 95 >> http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: >> Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: >> DE163214194 >> > From mika at async.caltech.edu Mon Dec 29 10:23:17 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 29 Dec 2008 01:23:17 -0800 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." Message-ID: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Jay writes: > ... > >I'm still not much of a believer in user threads. >Essentially I have always had kernel threads, with a processor-portable >interface, and one machine-wide scheduler..seems good.. > ... Except that on some machines, some operating systems, Modula-3's user threads are much more efficient (and if not, far more easily modifiable) than the provided kernel threads when you perform certain types of operations in certain situations. I often write programs with many (hundreds, thousands) of active threads and run into these sorts of problems frequently. You could argue that it should not be so... that every OS should provide kernel threads that are better than Modula-3's own user threads under all conceivable workloads, but what good does that do? Seems better to have a way of avoiding that can of worms by being able to control one's own implementation. Another reason that occurred to me when reading some very old M3 documentation is that there do occasionally exist systems that don't have a real operating system. There may also be other deficient systems that don't have pthreads. With an existing user-level threading library you can easily support such systems. (M3 has supported such systems in the past...) Mika From jay.krell at cornell.edu Mon Dec 29 10:47:26 2008 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Dec 2008 09:47:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> References: Your message of "Mon, 29 Dec 2008 08:47:02 GMT." <200812290923.mBT9NHGk031060@camembert.async.caltech.edu> Message-ID: I'm aware of systems without threads, e.g. MS-DOS or apparently many of the systems Modula-3 first ran on. (Really? pthreads wasn't very available back then?) The case of many threads, um.. You don't run out of address space on the stacks? I guess the Modula-3 stuff honors requests for very small stacks? (need to fix that it leaks a pthread_attr here..) Not a problem on 64bit systems. The large number of threads doesn't bog down scheduling? Is it hundreds/thousands of threads alive at the same time, or serially? Serially, no big deal. Modifiable, sure. The more you implement "yourself", the more you can change. You can hope for folks to build in some "configurability" and "tunability", but nothing beats the ability to change or replace the thing arbitrarily. Maybe having hundreds/thousands is "wrong"? I have repeatedly seen guidance against creating "many" threads. Better to have a "thread pool", and/or use "completion ports" (aka select?). Maybe you should have just a few threads, and then decide for yourself what each should run? You know, isn't the guidance to have roughly one thread per processor? You aren't going to scale beyond that..unless you dedicate a thread per slow i/o, which is wasteful. You know there is this whole thing where..too many cooks spoil the broth. Having multiple schedulers on the system means they won't do an ideal job. It is best to have one scheduler, that "knows everything" -- knows about all the threads and what they are waiting for. The cost of threads..is both low and high. - Jay ---------------------------------------- > To: jay.krell at cornell.edu > Date: Mon, 29 Dec 2008 01:23:17 -0800 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > Jay writes: >> > ... >> >>I'm still not much of a believer in user threads. >>Essentially I have always had kernel threads, with a processor-portable >>interface, and one machine-wide scheduler..seems good.. >> > ... > > Except that on some machines, some operating systems, Modula-3's > user threads are much more efficient (and if not, far more easily > modifiable) than the provided kernel threads when you perform certain > types of operations in certain situations. I often write programs > with many (hundreds, thousands) of active threads and run into these > sorts of problems frequently. > > You could argue that it should not be so... that every OS should > provide kernel threads that are better than Modula-3's own user > threads under all conceivable workloads, but what good does that > do? Seems better to have a way of avoiding that can of worms by > being able to control one's own implementation. > > Another reason that occurred to me when reading some very old > M3 documentation is that there do occasionally exist systems > that don't have a real operating system. There may also be > other deficient systems that don't have pthreads. With an existing > user-level threading library you can easily support such systems. > (M3 has supported such systems in the past...) > > Mika From jay.krell at cornell.edu Tue Dec 30 03:07:24 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:07:24 +0000 Subject: [M3devel] current tinderbox status, broken by me? Message-ID: NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho. - Jay From stsp at elego.de Tue Dec 30 03:16:46 2008 From: stsp at elego.de (Stefan Sperling) Date: Tue, 30 Dec 2008 03:16:46 +0100 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081229203344.692E1B04044@birch.elegosoft.com> References: <20081229203344.692E1B04044@birch.elegosoft.com> Message-ID: <20081230021646.GA8558@jack.stsp.name> On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/29 21:33:44 Hey Jay, > Log message: > enough I386_OPENBSD support to build a bootstrap package, > that builds on the target, and results in a cm3 that > runs and correct errors out for lack of cm3.cfg, Cool, I'd like to play with this, once you have a bootstrap binary. Stefan From jay.krell at cornell.edu Tue Dec 30 03:28:09 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:28:09 +0000 Subject: [M3devel] regular cross builds, package format? Message-ID: So..it is reasonably easy to do a cross build, that outputs a bunch of assembly files, some C files, a "script" or makefile to build them. What you (I) do with such output is copy it to the target machine and run the script, which requires a C compiler, linker, and whatever libraries, and whatever "script runner" (sh, make, etc.). I've been using this stuff to bring up new platforms. It works well enough. There are a number of obvious questions. Should the tinderbox go ahead and do that, for "every" target platform? ("every" defined as known working ones) What should the form of the "cross build archive" be? Or, for that matter, what should be form of "native distributions"? Should it be possible to build one "intermediate" distribution that can be "finished" on either any autoconf-able Posix platform, or any supported platform? That is, should the platform-specificity be factored out of the intermediate representation, and the cross or universal or even native (unlikely) format be that of portable intermediate files, and the "finish" step includes building the backend (not fast). I gather this last thing was done by PM3, though I also gather it isn't a small task. (I need to look at their code.) Platform-specifity leaks into the intermediate representation presumably a lot. You know, the size of integers, alignment affects on layout, etc. As well, if some systems use RTOSmmap.c and others RTOSbr.c -- or even .m3 files, the "universal" distribution would have to carry both files and delay the choice. If you reject the "universal" notion, there remain some questions, though less complicated ones. Simpple ones like what the "finish script" is. I generate a makefile, but it seems to be buggy. Sometimes there is no incrementality, it builds everything every time. Sometimes it generates a warning for each file. I don't know why, but since these are so far only for my use, ok. Of course there remains the question of cminstall or no cminstall. I've been getting by without cminstall for quite some time across a few platforms. A "universal" distribution would probably chose neither of the current options, but something that works better like autoconf. I'm not a huge fan of autoconf, but it has some very tempting characteristics, where it can sniff out system behavior as long as it is either in a known set or not too far off. - Jay From jay.krell at cornell.edu Tue Dec 30 03:57:12 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 02:57:12 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081230021646.GA8558@jack.stsp.name> References: <20081229203344.692E1B04044@birch.elegosoft.com> <20081230021646.GA8558@jack.stsp.name> Message-ID: Cool. You can try http://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-I386_OPENBSD-1.tar.gz, but hopefully soon I'll have "std" and "min". The boot archive builds cm3 only, no libs, no cm3cg -- though it should be enough to build them natively (plus a current source tree). If you use this, be aware that something is a little off in my bootstrapping process in that none of do-cm3-std do-cm3-min upgrade etc., seem quite right. upgrade in particular requires pkg/m3core and pkg/libm3. None of the sets imply TO ME building both m3cc and cm3, though "front" might actually (it ought not build m3cc). Also, the sorting to build in dependency order in pylib.py seems to not be working, strange. Anyway, this is fairly minor. There is also a definite possiblity of cross-building a complete distribution, you just have to build GNU cc (similar to building m3cg), GNU as, GNU ld, and either copy or build the C runtime and startup files...at least for *_*BSD, *_LINUX, SOLgnu, I386_CYGWIN, I don't know about the others (Darwin does not use GNU ld nor I think GNU as, whatever it does use, I don't know if it is easily re-hosted, similar SOLsun.) (Tony, please speak up if you really meant X86_* and not I386_*.) - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 03:16:46 +0100 > From: stsp at elego.de > To: jkrell at elego.de > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] [M3commit] CVS Update: cm3 > > On Mon, Dec 29, 2008 at 09:33:44PM +0000, Jay Krell wrote: >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/29 21:33:44 > > Hey Jay, > >> Log message: >> enough I386_OPENBSD support to build a bootstrap package, >> that builds on the target, and results in a cm3 that >> runs and correct errors out for lack of cm3.cfg, > > Cool, I'd like to play with this, once you have a bootstrap binary. > > Stefan From dabenavidesd at yahoo.es Tue Dec 30 07:01:45 2008 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 30 Dec 2008 06:01:45 +0000 (GMT) Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: Message-ID: <908914.75062.qm@web28601.mail.ukl.yahoo.com> In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? I would like to hear your comments on that, let me know what else is missing to know the FilePosix is safe. Thanks in advance, --- El lun, 29/12/08, Jay wrote: De: Jay Asunto: [M3devel] current tinderbox status, broken by me? Para: "m3devel" Fecha: lunes, 29 diciembre, 2008 9:07 NEXT Fatal Error: bad version stamps: SocketPosix.m3 4678 4679 version stamp mismatch: Compiler.Platform 4680 => SocketPosix.m3 4681 => Compiler.i3 4682 version stamp mismatch: Compiler.ThisPlatform 4683 => SocketPosix.m3 4684 => Compiler.i3 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16? failed *** 4686 compile return value: 0 4687 [end compile 2008.12.30 02:25:17] 4688 *** COMPILE FAILED No More Errors 1) Is it reasonable to suggest that the compiler output the source to clashing definitions? Or is that a bogus naive suggestion? It is probably legitimate, at least to identify the files in question. Could be that not enough information is around to identify "what changed", just some "summary hash". 2) Is it really broken? I did go slightly round and round on this. I had another new platform defined in one or another file and then when I backed down and just did one reasonly finished platform, I forgot at first to cleanup all the debris. But it certainly worked when I commited. I'm progressing on the native build, so if it is broken, I expect to run into it myself. 3) Is it reasonable to annotate a type with some sort of "relax, additions are safe" annotation? Probably not. It's probably too gray and more complicated than that, since additions in general are not as safe as people thing. Adding to a record -- folks might be copying based on the size and might get back an older smaller version. Adding to an enum, folks might be switching on it and expect to have the range covered. Adding a function to an interface though, probably ok. Not the case here, and perhaps not something the compiler even complains about. One that that might be eek-out-able is adding to an enum, and such annotated types, switches on them must have an else clause, and they must not be used for array indexing? Eh, maybe just leave it alone. Anything else? Is this just basically how it has to be? OR, how about, can the duplicity be eliminated? Like, can Compiler.i3 be fully generated by the compiler? Similarly, Csetjmp.i3 should be generated by the compiler, eh? Or at least partly? You know, the size and alignment of the jmp_buf is recorded in both Csetjmp.i3 and Target.i3. Duplication is generally bad imho -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Dec 30 08:48:17 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:48:17 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <908914.75062.qm@web28601.mail.ukl.yahoo.com> References: <908914.75062.qm@web28601.mail.ukl.yahoo.com> Message-ID: The C code I introduced should behave identically to or better than the Modula-3 code it replaced. The Modula-3 code dependend on tedious error-prone rewriting of C headers in Modula-3. Mistakes there easily lead to the overall safety of the system being violated. I believe the C code adheres to the safety guarantee below. Can you check? - Jay ________________________________ > Date: Tue, 30 Dec 2008 06:01:45 +0000 > From: dabenavidesd at yahoo.es > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] current tinderbox status, broken by me? > > In the case of adding procedures not in all the cases is Ok for INTERFACEs as pointed in the SPwM3 (green book, Chapter 6, page 156), also online available from ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-053.pdf " (page 26) "a module exporting a safe INTERFACE must guarantee that 'no programming error' by a safe client of that interface can lead to an unchecked runtime error." > Following the same idea exposed there, is possible to call the RegularFile functions of FilePosix Interface and get runtime errors? which are the guarantees that protect from causing an unchecked runtime error? > For instance a buggy code could use concurrently threads locking when writing the files concurrently, it will make a mess in the file and so may hole/leaks in memory? (could the file could be an IO port like parallel or serial)? > I would like to > hear your comments on that, let me know what else is missing to know the FilePosix is safe. > Thanks in advance, > > --- El lun, 29/12/08, Jay wrote: > De: Jay > Asunto: [M3devel] current tinderbox status, broken by me? > Para: "m3devel" > Fecha: lunes, 29 diciembre, 2008 9:07 > > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix..m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 > -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to clashing > definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I > had another new platform defined in one or another file and then when I > backed down and just did one reasonly finished platform, I forgot at first to > cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to run > into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, additions > are safe" annotation? > Probably not. It's probably too gray and more complicated than that, since > additions in general are not as safe as people thing. Adding to a record -- > folks might be copying based on the size and might get back an older smaller > version. Adding to an enum, folks might be switching on it and expect to have > the range covered. Adding a function to an interface though, probably ok. Not > the case here, and perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an > enum, and such annotated > types, switches on them must have an else clause, and they must not be used for > array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf is > recorded in both Csetjmp.i3 and Target..i3. Duplication is generally bad imho > > From jay.krell at cornell.edu Tue Dec 30 08:51:56 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 07:51:56 +0000 Subject: [M3devel] darwin-generic waitpid? Message-ID: It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and having a C wrapper do the translation. As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. - Jay From jay.krell at cornell.edu Tue Dec 30 09:34:03 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 08:34:03 +0000 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the waitpid(nohang)/sleep() loop that I had removed in other places, alas -- replaced in pthreads by waitpid(hang), whereas userthreads still has waitpid(nohang)/sleep (another flaw in user threads?) and..isn't this a little odd: RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); If there are no users of this, I'll just comment it out. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; m3devel at elegosoft.com > Date: Tue, 30 Dec 2008 07:51:56 +0000 > Subject: [M3devel] darwin-generic waitpid? > > > It looks like waitpid's output is endian-dependent, and that therefore darwin-generic is wrong on some architectures, asis linux-common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result bitfields with uint8_t and > having a C wrapper do the translation. > > > As well, for increased portability, the flags will be input uint8_t fields in the record, rather than platform-dependent flags. > > > - Jay From wagner at elegosoft.com Tue Dec 30 11:01:24 2008 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 30 Dec 2008 11:01:24 +0100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 for details. It occurs whenever you add or remove target platforms. It is exactly what the last-ok build was made for :-) As for your other comments, I would appreciate if the compiler would output the exact conflicting file paths, but I believe that the information is not easily available at that point. Olaf Quoting Jay : > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship > -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? > -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? > -DCM3_LAST_CHANGED=?2008-03-16? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Dec 30 11:04:23 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:04:23 +0000 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> References: <20081230110124.qnzl03m5lcscsgo0@mail.elegosoft.com> Message-ID: Ah, I thought the Tinderbox always ran upgrade.sh first. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 11:01:24 +0100 > From: wagner at elegosoft.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] current tinderbox status, broken by me? > > This is a known problem; see http://www.opencm3.net/known-problems.html#p_1_5 > for details. It occurs whenever you add or remove target platforms. > > It is exactly what the last-ok build was made for :-) > > As for your other comments, I would appreciate if the compiler would > output the exact conflicting file paths, but I believe that the > information is not easily available at that point. > > Olaf > > Quoting Jay : > >> >> NEXT Fatal Error: bad version stamps: SocketPosix.m3 >> 4678 >> 4679 version stamp mismatch: Compiler.Platform >> 4680 => SocketPosix.m3 >> 4681 => Compiler.i3 >> 4682 version stamp mismatch: Compiler.ThisPlatform >> 4683 => SocketPosix.m3 >> 4684 => Compiler.i3 >> 4685 NEXT *** execution of cm3 -build >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? && cm3 -ship >> -DROOT=?/home/m3/work/cm3-ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? >> -DCM3_VERSION_TEXT=?d5.7.0? -DCM3_VERSION_NUMBER=?050700? >> -DCM3_LAST_CHANGED=?2008-03-16? failed *** >> 4686 compile return value: 0 >> 4687 [end compile 2008.12.30 02:25:17] >> 4688 *** COMPILE FAILED >> No More Errors >> >> >> 1) >> Is it reasonable to suggest that the compiler output the source to >> clashing definitions? >> Or is that a bogus naive suggestion? >> It is probably legitimate, at least to identify the files in question. >> Could be that not enough information is around to identify "what >> changed", just some "summary hash". >> >> >> 2) Is it really broken? >> I did go slightly round and round on this. >> I had another new platform defined in one or another file and then >> when I backed down and just did one reasonly finished platform, I >> forgot at first to cleanup all the debris. >> But it certainly worked when I commited. >> I'm progressing on the native build, so if it is broken, I expect to >> run into it myself. >> >> >> 3) Is it reasonable to annotate a type with some sort of "relax, >> additions are safe" annotation? >> Probably not. It's probably too gray and more complicated than that, >> since additions in general are not as safe as people thing. Adding >> to a record -- folks might be copying based on the size and might >> get back an older smaller version. Adding to an enum, folks might be >> switching on it and expect to have the range covered. Adding a >> function to an interface though, probably ok. Not the case here, and >> perhaps not something the compiler even complains about. >> One that that might be eek-out-able is adding to an enum, and such >> annotated types, switches on them must have an else clause, and they >> must not be used for array indexing? >> Eh, maybe just leave it alone. >> >> >> Anything else? Is this just basically how it has to be? >> OR, how about, can the duplicity be eliminated? >> Like, can Compiler.i3 be fully generated by the compiler? >> Similarly, Csetjmp.i3 should be generated by the compiler, eh? >> Or at least partly? You know, the size and alignment of the jmp_buf >> is recorded in both Csetjmp.i3 and Target.i3. Duplication is >> generally bad imho > > > > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > From jay.krell at cornell.edu Tue Dec 30 11:08:53 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 10:08:53 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: I wish: 1) you could put declarations in .m3 files 2) could be entirely optional, as long as the C functions are named Module__Function. There are three implementations of RTOS.GetMemory. In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. - Jay ---------------------------------------- > Date: Tue, 30 Dec 2008 08:00:29 +0000 > To: m3commit at elegosoft.com > From: jkrell at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: jkrell at birch. 08/12/30 08:00:29 > > Modified files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 > FilePosixC.c m3makefile > Added files: > cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 > > Log message: > put the C functions in their own interface, and other small change > (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) > From jay.krell at cornell.edu Tue Dec 30 18:38:33 2008 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Dec 2008 17:38:33 +0000 Subject: [M3devel] FW: [M3commit] <*external*> In-Reply-To: <20081230070029.A1A2410D5C41@birch.elegosoft.com> References: <20081230070029.A1A2410D5C41@birch.elegosoft.com> Message-ID: [was truncated, and now edited slightly] > I wish: > > > 1) you could put declarations in .m3 files > > > 2) could be entirely optional, as long as the C functions are named Module__Function. > > > There are three implementations of RTOS.GetMemory. > In order to write one of them in C, I "had to" write all of them, because I had to mark it external and then make all the implementations external. > > > - Jay > > ---------------------------------------- >> Date: Tue, 30 Dec 2008 08:00:29 +0000 >> To: m3commit at elegosoft.com >> From: jkrell at elego.de >> Subject: [M3commit] CVS Update: cm3 >> >> CVSROOT: /usr/cvs >> Changes by: jkrell at birch. 08/12/30 08:00:29 >> >> Modified files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosix.i3 FilePosix.m3 >> FilePosixC.c m3makefile >> Added files: >> cm3/m3-libs/libm3/src/os/POSIX/: FilePosixC.i3 >> >> Log message: >> put the C functions in their own interface, and other small change >> (e.g. typedef ptrdiff_t INTEGER instead of equivalent other statement) >> From hosking at cs.purdue.edu Wed Dec 31 00:47:46 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:47:46 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: Message-ID: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay From hosking at cs.purdue.edu Wed Dec 31 00:48:43 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:48:43 +1100 Subject: [M3devel] and sysutils waitpid In-Reply-To: References: Message-ID: Yes, this definitely needs cleanup. On 30 Dec 2008, at 19:34, Jay wrote: > > On a related note...C:\dev2\cm3.2\m3-libs\sysutils\src\POSIX has the > waitpid(nohang)/sleep() loop that I had removed in other places, > alas -- replaced in pthreads by waitpid(hang), whereas userthreads > still has waitpid(nohang)/sleep (another flaw in user threads?) > > and..isn't this a little odd: > > RETURN MIN(LAST(Process.ExitCode),LOOPHOLE(statusM3,Uexec.w_A)); > > If there are no users of this, I'll just comment it out. > > - Jay > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: hosking at cs.purdue.edu; m3devel at elegosoft.com >> Date: Tue, 30 Dec 2008 07:51:56 +0000 >> Subject: [M3devel] darwin-generic waitpid? >> >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. >> >> >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay From hosking at cs.purdue.edu Wed Dec 31 00:50:35 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 10:50:35 +1100 Subject: [M3devel] current tinderbox status, broken by me? In-Reply-To: References: Message-ID: These result from inconsistency between the platforms defined statically in cm3 and the platforms declared in m3core. You need to use the version in m3core that matches what you had when building cm3. Then, extend cm3 before compiling the new m3core version. On 30 Dec 2008, at 13:07, Jay wrote: > > NEXT Fatal Error: bad version stamps: SocketPosix.m3 > 4678 > 4679 version stamp mismatch: Compiler.Platform > 4680 => SocketPosix.m3 > 4681 => Compiler.i3 > 4682 version stamp mismatch: Compiler.ThisPlatform > 4683 => SocketPosix.m3 > 4684 => Compiler.i3 > 4685 NEXT *** execution of cm3 -build -DROOT=?/home/m3/work/cm3- > ws/birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? && cm3 -ship -DROOT=?/home/m3/work/cm3-ws/ > birch.elegosoft.com-2008-12-30-01-00-04/cm3? -DCM3_VERSION_TEXT= > ?d5.7.0? -DCM3_VERSION_NUMBER=?050700? -DCM3_LAST_CHANGED=?2008-03-16 > ? failed *** > 4686 compile return value: 0 > 4687 [end compile 2008.12.30 02:25:17] > 4688 *** COMPILE FAILED > No More Errors > > > 1) > Is it reasonable to suggest that the compiler output the source to > clashing definitions? > Or is that a bogus naive suggestion? > It is probably legitimate, at least to identify the files in question. > Could be that not enough information is around to identify "what > changed", just some "summary hash". > > > 2) Is it really broken? > I did go slightly round and round on this. > I had another new platform defined in one or another file and then > when I backed down and just did one reasonly finished platform, I > forgot at first to cleanup all the debris. > But it certainly worked when I commited. > I'm progressing on the native build, so if it is broken, I expect to > run into it myself. > > > 3) Is it reasonable to annotate a type with some sort of "relax, > additions are safe" annotation? > Probably not. It's probably too gray and more complicated than that, > since additions in general are not as safe as people thing. Adding > to a record -- folks might be copying based on the size and might > get back an older smaller version. Adding to an enum, folks might be > switching on it and expect to have the range covered. Adding a > function to an interface though, probably ok. Not the case here, and > perhaps not something the compiler even complains about. > One that that might be eek-out-able is adding to an enum, and such > annotated types, switches on them must have an else clause, and they > must not be used for array indexing? > Eh, maybe just leave it alone. > > > Anything else? Is this just basically how it has to be? > OR, how about, can the duplicity be eliminated? > Like, can Compiler.i3 be fully generated by the compiler? > Similarly, Csetjmp.i3 should be generated by the compiler, eh? > Or at least partly? You know, the size and alignment of the jmp_buf > is recorded in both Csetjmp.i3 and Target.i3. Duplication is > generally bad imho From jay.krell at cornell.edu Wed Dec 31 01:09:58 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 00:09:58 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: I'd like to avoid cloning any headers, and use clear portable C wrappers to bridge between Modula-3 and stuff written/declared in C. e.g. no bit fields. These are contrary goals. Or, at least, only clone headers that are essentially identical across all Posix platforms, or which the only variables are endianness and word size. There are a few possible compromises. Currently there is linux-common, openbsd-common. I would NOT like to add netbsd-common, freebsd-common, solaris-common. As such, I'm pushing stuff from "*-common" to plain "Common". As well, by not cloning header content that is unused in the cm3 tree, much is saved. Compare the number of lines in LINUXLIBC6 vs. linux-common. But I'd like to get further than that. Similarly, not cloning header content that doesn't present a portable interface, which tends to lead to it not being used, since most of the cm3 code is factored only on Posix vs. Win32, and not one Posix superset or another. I also believe that C is "everywhere" and will remain "everywhere". Except very constrained "safe" environments like a web browser or Java-only Google phone, where Modula-3 is about as far as being able to run as C is. I think C wrappers are very preferable to cloning headers. The endian-ness is actually abstracted out in normal C code. There's the various macros for cracking the 32 bit status. Ok, maybe that is apples and oranges. It is abstracted out in the client C code and the client Modula-3 code, except that Modula-3 client code uses LOOPHOLE. Both the C headers and Modula-3 headers are "messy", the C headers sometimes more so, such as to cater to other (BSD) interfaces. I realize there are the Win32 headers to be concerned with as well. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 10:47:46 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > On 30 Dec 2008, at 18:51, Jay wrote: > >> >> It looks like waitpid's output is endian-dependent, and that >> therefore darwin-generic is wrong on some architectures, asis linux- >> common and openbsd-common. >> >> >> I expect to fix this shortly, by replacing all the Modula-3 result >> bitfields with uint8_t and >> having a C wrapper do the translation. > > I am concerned that so much C wrapper stuff is starting to creep in. > I want to be able to see the endianness in the Modula-3 code, not have > it squirrelled away in C code. What if I don't have a C compiler? > Jay, please be careful with your assumptions about platforms always > having C. > >> As well, for increased portability, the flags will be input uint8_t >> fields in the record, rather than platform-dependent flags. >> >> >> - Jay > From hosking at cs.purdue.edu Wed Dec 31 01:47:48 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 31 Dec 2008 11:47:48 +1100 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Sure, I understand. I think what you are doing is great for improving the reliability of the C-interface code. Are you able to test all the platforms? I am worried we are going to have bugs introduced that someone (you!) will need to chase down. On 31 Dec 2008, at 11:09, Jay wrote: > > I'd like to avoid cloning any headers, and use clear portable C > wrappers to bridge between Modula-3 and stuff written/declared in C. > e.g. no bit fields. > > These are contrary goals. > > Or, at least, only clone headers that are essentially identical > across all Posix platforms, or which the only variables are > endianness and word size. > > There are a few possible compromises. > Currently there is linux-common, openbsd-common. > I would NOT like to add netbsd-common, freebsd-common, solaris-common. > As such, I'm pushing stuff from "*-common" to plain "Common". > > As well, by not cloning header content that is unused in the cm3 > tree, much is saved. > Compare the number of lines in LINUXLIBC6 vs. linux-common. > But I'd like to get further than that. > > Similarly, not cloning header content that doesn't present a > portable interface, which tends to lead to it not being used, since > most of the cm3 code is factored only on Posix vs. Win32, and not > one Posix superset or another. > > I also believe that C is "everywhere" and will remain "everywhere". > Except very constrained "safe" environments like a web browser or > Java-only Google phone, where Modula-3 is about as far as being able > to run as C is. > > I think C wrappers are very preferable to cloning headers. > > The endian-ness is actually abstracted out in normal C code. > There's the various macros for cracking the 32 bit status. > Ok, maybe that is apples and oranges. It is abstracted out in the > client C code and the client Modula-3 code, except that Modula-3 > client code uses LOOPHOLE. Both the C headers and Modula-3 headers > are "messy", the C headers sometimes more so, such as to cater to > other (BSD) interfaces. > > I realize there are the Win32 headers to be concerned with as well. > > - Jay > > ---------------------------------------- >> From: hosking at cs.purdue.edu >> To: jay.krell at cornell.edu >> Date: Wed, 31 Dec 2008 10:47:46 +1100 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] darwin-generic waitpid? >> >> On 30 Dec 2008, at 18:51, Jay wrote: >> >>> >>> It looks like waitpid's output is endian-dependent, and that >>> therefore darwin-generic is wrong on some architectures, asis linux- >>> common and openbsd-common. >>> >>> >>> I expect to fix this shortly, by replacing all the Modula-3 result >>> bitfields with uint8_t and >>> having a C wrapper do the translation. >> >> I am concerned that so much C wrapper stuff is starting to creep in. >> I want to be able to see the endianness in the Modula-3 code, not >> have >> it squirrelled away in C code. What if I don't have a C compiler? >> Jay, please be careful with your assumptions about platforms always >> having C. >> >>> As well, for increased portability, the flags will be input uint8_t >>> fields in the record, rather than platform-dependent flags. >>> >>> >>> - Jay >> From jay.krell at cornell.edu Wed Dec 31 02:25:31 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 01:25:31 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: I am not able to test everything this week, on vacation, just have two laptops with me, and not quite /everything/ at home yet, but I have available a lot now and am still working on getting and setting up more hardware and operating systems, and maybe will use virtualization more (I386_OPENBSD is in VirtualPC, Solaris/x86 in VirtualBox bugchecks the host too frequently(. I'm not sure the inventory is important or impressive or embarassing. :) I've got a variety of MIPS64 (SGI), PPC64 (RS/6000), PPC32 (Apple), ARM (Western Digital MyBook "World Edition), IA64 (HP Integrity), SPARC64 (Sun), AMD64 (Sun and PCs), running variety of Linux, OpenBSD, Darwin, AIX, not currently NetBSD but will, NT, Solaris.. Alphas are on the way, HP-PA might be on the way, the main missing piece is PPC64_DARWIN hardware. :) - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 05:32:25 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 04:32:25 +0000 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> <3703BB74-7E17-4370-BBDC-B59BAF40F6AA@cs.purdue.edu> Message-ID: Tony, Hi, sorry, I'm a little confused. Beware C wrappers -- don't write them. That's what it sounded like at first. Or: Beware C wrappers -- be careful writing them. ? On the second, well, yeah, ok. I haven't actually yet improved much -- since I stick to "new" ports. "Soon" I might take larger steps. I'll probably first cave and add user thread support to {PPC,AMD64,SPARC{32,64}}_LINUX, I386_SOLARIS, *_OPENBSD, etc., while continuing to minimize or eliminate header clones. And then maybe move LINUXLIBC6 to linux-common/common. I should say..um..IF your concern is just "be careful writing C code"... I wonder how far I should go. I'm obessive. I keep looking at linux-common/openbsd-common/cygwin and I wonder what I can remove. And how. Removing unused stuff, easy. Wrapping functions that actually are functions, not too bad. But types and constants..you can try to write functions to take over all their uses, but I don't think that will do for all of them. For example, struct flock I removed. The "worst" cases are probably: Uerrno Upthread In particular, these are the files for which C wrapping is likely to do the "most damage" with respect to efficiency and code structure. Uerrno..the way to wrap it in C is, well..one thing I do NOT want to do is go through and make up our own errno values and map back and forth everywhere. Yuck. However, one thing that seems half reasonable is to have GetEAGAIN(), GetEINVALID(), GetENOMEM(), etc. or a bulk struct { int EINVAL, ENOMEM, etc. } * GetErrnoValues(). The second version is "less chatty" but risky, like if the struct gets copied and changes size through time (versioning). The first is much safer. In both cases, the errors loses their const-ness and you can, I assume, no longer switch/case on them. Now, error checking..efficiency..probably ok -- code that is dealing with errors doesn't have to be fast. Change the switches/cases to if/elseif/elseif ladders. I think I will do this. Upthread, here I think part of the answer is that all of the functions have identical prototypes. So "half" the file just goes to Common and done, no C wrapping. That leaves the types. sem_t, which is vaguely related, there is just one of. I think we can just declare that variable in C, since you don't like the over allocation/alignment. But that's an unusual benefit -- something there is only one of. For types that are created repeatedly..well, it depends. I have to look into the allocator interface.. if the objects are heap allocated, one thing you can maybe do is like: RTAllocator(Upthread.GetSize_rwlock(), Upthread.GetAlign_rwlock()). Assuming size and alignment are parameters, or assuming that alignment is always maximal. If the objects are stack allocated, well, that breaks down. You /could/ heap allocate them always, but that's a sort of inefficiency I'm reluctant to introduce. I don't mind turning constants, esp. seldom used ones, into function calls, or copying small structs (stat), but turning stack allocation into heap allocation, yuck. Another answer is to leave these alone. Generate Uerrno.i3 from Uerrno.i3.cpp and leave it that. Maybe generate them as a matter of course in any /native/ build and compare to the "checked in" copy. Well, um, native builds also benefit from being able to run tests, so you can accomplish the same thing there -- pass the constants, type sizes, etc. to C code that asserts equivalence. Upthread.i3 could be similar. To address the types vs. prototypes, I'd say move the types, and probably all the initializers, into a per-system Upthreadtypes.i3, or Usysdep.i3. Usocket.i3 is similar -- identical prototypes, but constants vary. Move the constants into "Usysdep.i3"? Or use accessor functions? I have to look at the code more besides. I don't have a plan. Anyway, at the very least, we have: jay at ppca:/dev2/cm3/m3-libs/m3core/src/unix$ wc -l linux-common/*.i3 415 total openbsd-common => 516 wc -l linux-libc6/*.i3 4011 Not bad to be at 10%, and user threads won't regress all that much. Maybe stop there? Well, Uutsname.i3 is probably going away. It has just one user. - Jay ---------------------------------------- > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Wed, 31 Dec 2008 11:47:48 +1100 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] darwin-generic waitpid? > > Sure, I understand. I think what you are doing is great for improving > the reliability of the C-interface code. Are you able to test all the > platforms? I am worried we are going to have bugs introduced that > someone (you!) will need to chase down. > > On 31 Dec 2008, at 11:09, Jay wrote: > >> >> I'd like to avoid cloning any headers, and use clear portable C >> wrappers to bridge between Modula-3 and stuff written/declared in C. >> e.g. no bit fields. >> >> These are contrary goals. >> >> Or, at least, only clone headers that are essentially identical >> across all Posix platforms, or which the only variables are >> endianness and word size. >> >> There are a few possible compromises. >> Currently there is linux-common, openbsd-common. >> I would NOT like to add netbsd-common, freebsd-common, solaris-common. >> As such, I'm pushing stuff from "*-common" to plain "Common". >> >> As well, by not cloning header content that is unused in the cm3 >> tree, much is saved. >> Compare the number of lines in LINUXLIBC6 vs. linux-common. >> But I'd like to get further than that. >> >> Similarly, not cloning header content that doesn't present a >> portable interface, which tends to lead to it not being used, since >> most of the cm3 code is factored only on Posix vs. Win32, and not >> one Posix superset or another. >> >> I also believe that C is "everywhere" and will remain "everywhere". >> Except very constrained "safe" environments like a web browser or >> Java-only Google phone, where Modula-3 is about as far as being able >> to run as C is. >> >> I think C wrappers are very preferable to cloning headers. >> >> The endian-ness is actually abstracted out in normal C code. >> There's the various macros for cracking the 32 bit status. >> Ok, maybe that is apples and oranges. It is abstracted out in the >> client C code and the client Modula-3 code, except that Modula-3 >> client code uses LOOPHOLE. Both the C headers and Modula-3 headers >> are "messy", the C headers sometimes more so, such as to cater to >> other (BSD) interfaces. >> >> I realize there are the Win32 headers to be concerned with as well. >> >> - Jay >> >> ---------------------------------------- >>> From: hosking at cs.purdue.edu >>> To: jay.krell at cornell.edu >>> Date: Wed, 31 Dec 2008 10:47:46 +1100 >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] darwin-generic waitpid? >>> >>> On 30 Dec 2008, at 18:51, Jay wrote: >>> >>>> >>>> It looks like waitpid's output is endian-dependent, and that >>>> therefore darwin-generic is wrong on some architectures, asis linux- >>>> common and openbsd-common. >>>> >>>> >>>> I expect to fix this shortly, by replacing all the Modula-3 result >>>> bitfields with uint8_t and >>>> having a C wrapper do the translation. >>> >>> I am concerned that so much C wrapper stuff is starting to creep in. >>> I want to be able to see the endianness in the Modula-3 code, not >>> have >>> it squirrelled away in C code. What if I don't have a C compiler? >>> Jay, please be careful with your assumptions about platforms always >>> having C. >>> >>>> As well, for increased portability, the flags will be input uint8_t >>>> fields in the record, rather than platform-dependent flags. >>>> >>>> >>>> - Jay >>> > From jay.krell at cornell.edu Wed Dec 31 06:33:22 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 05:33:22 +0000 Subject: [M3devel] variations of waitpid..? Message-ID: variations of waitpid.. Hey, we already have, that I put in on my initiative, months ago: C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) VAR result: int; statusM3: Uwaitpid.waitpid_status_t; CONST Delay = 0.1D0; BEGIN LOOP result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Pause(Delay); END; 0 *> RETURN statusM3.w_Loophole; END WaitProcess; pthreads and cygwin: C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 and C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 PROCEDURE WaitProcess (pid: int): int = (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) VAR statusM3: Uwaitpid.waitpid_status_t; BEGIN LOOP WITH r = Uwaitpid.waitpid(pid, statusM3) DO IF r> 0 THEN RETURN statusM3.w_Loophole; END; < 0*> END; END; END WaitProcess; win32: PROCEDURE WaitProcess (pid: int): int = BEGIN RETURN 0; END WaitProcess; ThreadPThread and WaitProcessCygwin should share code, but that's not the point. However, now, to fix sysutils, we need something LIKE: UNSAFE INTERFACE SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; BEGIN END SystemPosixWaitPid. pthreads: UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = BEGIN RETURN waitpid(pid, status, 0); END WaitPid; BEGIN END SystemPosixWaitPidEfficient. user threads: UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; FROM Ctypes IMPORT int; FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; FROM Utypes IMPORT pid_t; PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = VAR result: int; CONST Delay = 0.1D0; BEGIN LOOP result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); IF result # 0 THEN EXIT END; Thread.Pause(Delay) END; END WaitPid; BEGIN END SystemPosixWaitPidPause. questions: Aside: Where do I put "unsafe"? Relevant: Sysutils has to implement this itself, right? Due to bootstrapping concerns. But we should still put /something/ in m3core, right? So then the big question, what to call it? Probably, the existing WaitProcess that does more than this WaitPid, should be scaled back and only do what WaitPid does? I will likely commit exactly what is shown above. The new interface and modules are in sysutils. The interface is private. The unfinished part is putting something in m3core, that sysutils could use, if not for boostrapping issuers, that it could use in the future. I kind of thing something a bit more is needed. You know, the Win32 implementation that asserts(false) is suspicious. At the very least, it should be removed. I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 interfaces? I also suspect part of the problem is that sysutils implements to a large extent the same abstractions as m3core and/or libm3, but with variations, causing it to be unable to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions maybe are not general/featureful enough. (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) ?? - Jay From jay.krell at cornell.edu Wed Dec 31 08:13:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 07:13:02 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: References: Message-ID: Hm. How about: PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? ? true for pthreads/win32/cygwin false for userthreads duplicated privately in sysutils to avoid depending on newer m3core like I did for the portable C-wrapped waitpid ? That way..the other code can do whatever it already is doing, no need to come up with abstractions for looping with nohang, vs. that plus the status loophole. The older fix I put it can be shrunk. The name of this is debatable, but I think the minimal level of abstraction is goodness. The name is actually perhaps too..complicated..like, what does it mean? Who uses it? What do they do with it? It means that waiting on a process to end allows threads in the waiting process to run. Perhaps it is too low level. - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Wed, 31 Dec 2008 05:33:22 +0000 > Subject: [M3devel] variations of waitpid..? > > > variations of waitpid.. > > Hey, we already have, that I put in on my initiative, months ago: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.m3 calls Pause(). *) > VAR > result: int; > statusM3: Uwaitpid.waitpid_status_t; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); > IF result # 0 THEN EXIT END; > Pause(Delay); > END; > 0 *> > RETURN statusM3.w_Loophole; > END WaitProcess; > > pthreads and cygwin: > > C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 > and > C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 > > PROCEDURE WaitProcess (pid: int): int = > (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) > VAR > statusM3: Uwaitpid.waitpid_status_t; > BEGIN > LOOP > WITH r = Uwaitpid.waitpid(pid, statusM3) DO > IF r> 0 THEN > RETURN statusM3.w_Loophole; > END; > < 0*> > END; > > END; > END WaitProcess; > > win32: > PROCEDURE WaitProcess (pid: int): int = > BEGIN > > RETURN 0; > END WaitProcess; > > ThreadPThread and WaitProcessCygwin should share code, but that's not the point. > > However, now, to fix sysutils, we need something LIKE: > > UNSAFE INTERFACE SystemPosixWaitPid; > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; > > BEGIN > END SystemPosixWaitPid. > > pthreads: > > UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > BEGIN > RETURN waitpid(pid, status, 0); > END WaitPid; > > BEGIN > END SystemPosixWaitPidEfficient. > > user threads: > > UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; > > FROM Ctypes IMPORT int; > FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; > FROM Utypes IMPORT pid_t; > > PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = > VAR result: int; > CONST Delay = 0.1D0; > BEGIN > LOOP > result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); > IF result # 0 THEN > EXIT > END; > Thread.Pause(Delay) > END; > END WaitPid; > > BEGIN > END SystemPosixWaitPidPause. > > questions: > > Aside: Where do I put "unsafe"? > > Relevant: Sysutils has to implement this itself, right? > Due to bootstrapping concerns. > > But we should still put /something/ in m3core, right? > > So then the big question, what to call it? > > Probably, the existing WaitProcess that does more than this WaitPid, should be scaled > back and only do what WaitPid does? > > I will likely commit exactly what is shown above. > The new interface and modules are in sysutils. The interface is private. > > The unfinished part is putting something in m3core, that sysutils could use, if > not for boostrapping issuers, that it could use in the future. > > I kind of thing something a bit more is needed. > You know, the Win32 implementation that asserts(false) is suspicious. > At the very least, it should be removed. > > I suspect I am missing something though, as suggested by Win32 having an assert(false) implementation. > Either that needs to be removed, or somehow the code can be unified, using just the existing m3core/libm3 > interfaces? > > I also suspect part of the problem is that sysutils implements to a large extent > the same abstractions as m3core and/or libm3, but with variations, causing it to be unable > to build on what is there, but having to reinvent..which suggests the m3core/libm3 abstractions > maybe are not general/featureful enough. > > (Btw, unstated above, is that sysutils/m3makefile will have the same sort of switch as m3core does, wrt if it builds for user/alaram threads or kernel/pthreads.) > > > ?? > > - Jay From mika at async.caltech.edu Wed Dec 31 08:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 30 Dec 2008 23:22:41 -0800 Subject: [M3devel] variations of waitpid..? In-Reply-To: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." Message-ID: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> I don't think it should be possible to stop other threads if the runtime could even conceivably support an environment where that's not necessary. You're introducing dependencies that will lead to (undetectable) deadlocks. People who declare their own <*EXTERNAL*> things are, as usual, on their own. Mika Jay writes: > >Hm. How about: > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? > >? >true for pthreads/win32/cygwin >false for userthreads > >duplicated privately in sysutils to avoid depending on newer m3core > like I did for the portable C-wrapped waitpid > >? > >That way..the other code can do whatever it already is doing, >no need to come up with abstractions for looping with nohang, >vs. that plus the status loophole. > >The older fix I put it can be shrunk. > >The name of this is debatable, but I think the minimal level of abstraction is > goodness. > >The name is actually perhaps too..complicated..like, what does it mean? >Who uses it? What do they do with it? > >It means that waiting on a process to end allows threads in the waiting proces >s to run. >Perhaps it is too low level. > > > - Jay > > > >---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Date: Wed, 31 Dec 2008 05:33:22 +0000 >> Subject: [M3devel] variations of waitpid..? >> >> >> variations of waitpid.. >> >> Hey, we already have, that I put in on my initiative, months ago: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix. >m3 calls Pause(). *) >> VAR >> result: int; >> statusM3: Uwaitpid.waitpid_status_t; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG); >> IF result # 0 THEN EXIT END; >> Pause(Delay); >> END; >> 0 *> >> RETURN statusM3.w_Loophole; >> END WaitProcess; >> >> pthreads and cygwin: >> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3 >> and >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3 >> >> PROCEDURE WaitProcess (pid: int): int = >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *) >> VAR >> statusM3: Uwaitpid.waitpid_status_t; >> BEGIN >> LOOP >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO >> IF r> 0 THEN >> RETURN statusM3.w_Loophole; >> END; >> < 0*> >> END; >> >> END; >> END WaitProcess; >> >> win32: >> PROCEDURE WaitProcess (pid: int): int = >> BEGIN >> >> RETURN 0; >> END WaitProcess; >> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po >int. >> >> However, now, to fix sysutils, we need something LIKE: >> >> UNSAFE INTERFACE SystemPosixWaitPid; >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int; >> >> BEGIN >> END SystemPosixWaitPid. >> >> pthreads: >> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> BEGIN >> RETURN waitpid(pid, status, 0); >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidEfficient. >> >> user threads: >> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid; >> >> FROM Ctypes IMPORT int; >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid; >> FROM Utypes IMPORT pid_t; >> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int = >> VAR result: int; >> CONST Delay = 0.1D0; >> BEGIN >> LOOP >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG); >> IF result # 0 THEN >> EXIT >> END; >> Thread.Pause(Delay) >> END; >> END WaitPid; >> >> BEGIN >> END SystemPosixWaitPidPause. >> >> questions: >> >> Aside: Where do I put "unsafe"? >> >> Relevant: Sysutils has to implement this itself, right? >> Due to bootstrapping concerns. >> >> But we should still put /something/ in m3core, right? >> >> So then the big question, what to call it? >> >> Probably, the existing WaitProcess that does more than this WaitPid, should >be scaled >> back and only do what WaitPid does? >> >> I will likely commit exactly what is shown above. >> The new interface and modules are in sysutils. The interface is private. >> >> The unfinished part is putting something in m3core, that sysutils could use, > if >> not for boostrapping issuers, that it could use in the future. >> >> I kind of thing something a bit more is needed. >> You know, the Win32 implementation that asserts(false) is suspicious. >> At the very least, it should be removed. >> >> I suspect I am missing something though, as suggested by Win32 having an ass >ert(false) implementation. >> Either that needs to be removed, or somehow the code can be unified, using j >ust the existing m3core/libm3 >> interfaces? >> >> I also suspect part of the problem is that sysutils implements to a large ex >tent >> the same abstractions as m3core and/or libm3, but with variations, causing i >t to be unable >> to build on what is there, but having to reinvent..which suggests the m3core >/libm3 abstractions >> maybe are not general/featureful enough. >> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth >reads.) >> >> >> ?? >> >> - Jay From mika at async.caltech.edu Wed Dec 31 09:22:41 2008 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 31 Dec 2008 00:22:41 -0800 Subject: [M3devel] TEXT & etc. Message-ID: <200812310822.mBV8Mf2j027994@camembert.async.caltech.edu> Hello everyone, I mentioned a week or two ago that I had run into horrific performance issues with CM3's TEXTs. It wasn't my intention to start a debate about Unicode and Mahjong characters, just to point out that there's a serious performance problem with CM3 in this area. I run into performance problems with CM3 from time to time because I am trying to maintain a fairly large amount of software written in Modula-3 and it appears to me that the SRC/PM3 compilers often pay better attention to performance issues than the newer CM3 compiler. Generally speaking I think Modula-3 needs to avoid getting into the Java situation: Java is squeezed between "fast enough" Python and "much more efficient" C++... Modula-3 I see as the only hope for getting the performance of C++ with the safety of Java. The problems I have seen with CM3 are in the following areas: 1. Thread mutex acquisition involving kernel calls (pthreads implementation) 2. ISTYPE and TYPECASE (appear much faster on PM3, don't know why, could just be a profiling issue). In any case TYPECASE is very slow because of having to search the type tree. This could be improved but I can't work on it because I always run into some kind of fatal problem trying to bootstrap the compiler. 3. The TEXT issue. I mentioned performance problems before. It turns out that it was showing up in another area. I upgraded my PowerBook a while back (to OS X 10.4) and of course everything stopped working, including at least half my Modula-3 programs. I thought it was a versioning problem with C libraries I was importing, but it turned out that in at least one case I was running out of stack space because of the calls to get_char on concatenated TEXTs. (Not very long ones, at that.) To get to the point. What follows is my new version of TextCat.m3, which I believe works, but I can't really say because I'm perennially incapable of bootstrapping CM3. The basic idea is that you can have *one* TextCat.T, at the "root" of your TEXT. It will flatten anything else. I don't see a simple way of modifying the CM3 TEXT implementation to do much better than this. Of course throwing out TextCat.T completely is an option. An advantage of what I've done is that I haven't changed any interfaces at all. So I don't get problems with version stamps, etc. Mika (* Copyright 1996-2000, Critical Mass, Inc. All rights reserved. *) (* See file COPYRIGHT-CMASS for details. *) UNSAFE MODULE TextCat EXPORTS RTHooks, TextCat; IMPORT TextClass, Text8, Text16; (* changed 12/31/2008: only return a T when neither of the arguments to Concat is already a T; otherwise flatten to a Text8 or Text16. Change affects only Concat and MultiCat *) REVEAL T = Public BRANDED "TextCat.T" OBJECT OVERRIDES get_info := MyGetInfo; get_char := MyGetChar; get_wide_char := MyGetWideChar; get_chars := MyGetChars; get_wide_chars := MyGetWideChars; END; PROCEDURE New (t, u: TEXT): TEXT = BEGIN RETURN Concat (t, u); END New; (* RTHooks.Concat -- called by the inline "&" operator *) PROCEDURE Concat (t, u: TEXT): TEXT = VAR ti, ui: TextClass.Info; BEGIN t.get_info (ti); IF (ti.length <= 0) THEN RETURN u; END; u.get_info (ui); IF (ui.length <= 0) THEN RETURN t; END; IF NOT ISTYPE(t,T) AND NOT ISTYPE(u,T) THEN RETURN NEW (T, a := t, b := u, a_len := ti.length, b_len := ui.length, a_or_b_wide := ti.wide OR ui.wide) ELSIF ti.wide OR ui.wide THEN WITH res = Text16.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_wide_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_wide_char(i) END; RETURN res END ELSE WITH res = Text8.Create(ti.length+ui.length) DO FOR i := 0 TO ti.length-1 DO res.contents[i] := t.get_char(i) END; FOR i := 0 TO ui.length-1 DO res.contents[ti.length+i] := t.get_char(i) END; RETURN res END END END Concat; PROCEDURE NewMulti (READONLY x: ARRAY OF TEXT): TEXT = BEGIN RETURN MultiCat (x); END NewMulti; (* RTHooks.MultiCat *) PROCEDURE MultiCat (READONLY x: ARRAY OF TEXT): TEXT = VAR wide := FALSE; length, start := 0; xi : TextClass.Info; BEGIN IF NUMBER (x) <= 0 THEN RETURN ""; END; IF NUMBER (x) = 1 THEN RETURN x[0]; END; IF NUMBER (x) = 2 THEN RETURN Concat(x[0],x[1]) END; FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); wide := wide OR xi.wide; length := length + xi.length END; IF wide THEN WITH res = Text16.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_wide_char(j) END; start := start + xi.length END; RETURN res END ELSE WITH res = Text8.Create(length) DO FOR i := FIRST(x) TO LAST(x) DO x[i].get_info (xi); FOR j := 0 TO xi.length-1 DO res.contents[start+j] := x[i].get_char(j) END; start := start + xi.length END; RETURN res END END END MultiCat; PROCEDURE MyGetInfo (t: T; VAR info: TextClass.Info) = BEGIN info.start := NIL; info.length := t.a_len + t.b_len; info.wide := t.a_or_b_wide; END MyGetInfo; PROCEDURE MyGetChar (t: T; index: CARDINAL): CHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetChar; PROCEDURE MyGetWideChar (t: T; index: CARDINAL): WIDECHAR = BEGIN IF (index < t.a_len) THEN RETURN t.a.get_wide_char (index); END; DEC (index, t.a_len); IF (index < t.b_len) THEN RETURN t.b.get_wide_char (index); END; DEC (index, t.b_len); index := -1; (* force a range fault *) <*NOWARN*> END MyGetWideChar; PROCEDURE MyGetChars (t: T; VAR a: ARRAY OF CHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetChars; PROCEDURE MyGetWideChars (t: T; VAR a: ARRAY OF WIDECHAR; start: CARDINAL) = VAR u: TEXT; a_offset, t_offset, u_offset: CARDINAL := 0; BEGIN u := t.a; IF (t_offset + t.a_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.a_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.a_len); u := t.b; IF (t_offset + t.b_len > start) THEN u_offset := MAX (start - t_offset, 0); u.get_wide_chars (SUBARRAY (a, a_offset, NUMBER (a) - a_offset), u_offset); INC (a_offset, t.b_len - u_offset); IF (a_offset >= NUMBER (a)) THEN RETURN; END; END; INC (t_offset, t.b_len); END MyGetWideChars; BEGIN END TextCat. From roland.illig at gmx.de Wed Dec 31 11:33:31 2008 From: roland.illig at gmx.de (Roland Illig) Date: Wed, 31 Dec 2008 11:33:31 +0100 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? Message-ID: <495B4A7B.6010505@gmx.de> Hi, when trying to build the simplest program, examples/hello, I get the following error messages: $ ~/usr.local/packages/cm3-5.4.0/bin/cm3 --- building in LINUXLIBC6 --- new source -> compiling Hello.m3 Hello.ms: Assembler messages: Hello.ms:21: Error: suffix or operands invalid for `push' Hello.ms:23: Error: suffix or operands invalid for `push' Hello.ms:27: Error: suffix or operands invalid for `pop' ... Is there some way of running cm3 on an x86_64 system? Roland From rcoleburn at scires.com Wed Dec 31 15:49:32 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 09:49:32 -0500 Subject: [M3devel] darwin-generic waitpid? In-Reply-To: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> References: <5C515925-88E0-40EB-A589-60545A27D257@cs.purdue.edu> Message-ID: <495B3F1E.1E75.00D7.1@scires.com> I agree with Tony about use of Modula-3 vs. C. Modula-3 is a systems programming language. We should be using it for systems programming, not C. My vote is to always use Modula-3, not C. Regards, Randy >>> Tony Hosking 12/30/2008 6:47 PM >>> On 30 Dec 2008, at 18:51, Jay wrote: > > It looks like waitpid's output is endian-dependent, and that > therefore darwin-generic is wrong on some architectures, asis linux- > common and openbsd-common. > > > I expect to fix this shortly, by replacing all the Modula-3 result > bitfields with uint8_t and > having a C wrapper do the translation. I am concerned that so much C wrapper stuff is starting to creep in. I want to be able to see the endianness in the Modula-3 code, not have it squirrelled away in C code. What if I don't have a C compiler? Jay, please be careful with your assumptions about platforms always having C. > As well, for increased portability, the flags will be input uint8_t > fields in the record, rather than platform-dependent flags. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:28:37 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:28:37 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> Message-ID: <495B4846.1E75.00D7.1@scires.com> Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:46:27 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:46:27 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> References: <20081231100311.8B4ED10D5DA1@birch.elegosoft.com> Message-ID: <495B4C74.1E75.00D7.1@scires.com> Jay: The "cm3-41-hack.c" file is an old file that isn't used anymore. I seem to recall that the CMass folks created it for me back in the days of v4.1 to get around a bug. When I committed the cm3ide package a while back, I also committed the WindowsResoures package. I kept the cm3-41-hack.c file in there for historical purposes just in case someone ever has occasion to run cm3 v4.1. Indeed, anyone who purchased the CMass "Reactor" product (as I did) has v4.1 of cm3. You will note in the WindowsResources m3makefile that the cm3-41-hack.c file is commented out and there is a long comment proceeding it to explain its purpose. On a cm3 v4.1 system, you would either need to uncomment this line, or fix the underlying bug that prompted the need for it in the first place. Regards, Randy >>> Jay Krell 12/31/2008 11:03 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:03:11 Modified files: cm3/m3-sys/windowsResources/src/: cm3-41-hack.c Log message: microoptimize and reformat -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 16:58:44 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 10:58:44 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20081231105714.24CF710D5DDE@birch.elegosoft.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> Message-ID: <495B4F55.1E75.00D7.1@scires.com> Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 18:59:03 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 12:59:03 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: <495B6B87.1E75.00D7.1@scires.com> Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy >>> 1/1/2009 5:25 AM >>> On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote: > > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation. It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine? > My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad. Except that the free download won't work on old versions of Windows. This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available. > I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents. > > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system. > > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own. It is not hard to write a linker in Modula-3. > From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment). What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice. > > Regards, > Randy -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:34:24 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:34:24 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:44:00 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:44:00 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B4F55.1E75.00D7.1@scires.com> References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy>>> Jay Krell 12/31/2008 11:57 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:57:14Added files:cm3/m3-sys/windowsResources/src/: m3overrides Log message:Add missing override to fix some builds.I'm highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:54:53 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:54:53 +0000 Subject: [M3devel] FW: [M3commit] CVS Update: cm3 In-Reply-To: <495B4846.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: truncated, around the cmd/bat stuff From: jay.krell at cornell.eduTo: rcoleburn at scires.com; m3devel at elegosoft.comSubject: RE: [M3devel] [M3commit] CVS Update: cm3Date: Wed, 31 Dec 2008 18:34:24 +0000 > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view.They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT.They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns.Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided.They are truly identical in every respect in the front end and back end.They only vary in the config file.This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have.Therefore, BUILD_DIR is arbitrary.I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so..I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't.Show me where it does.The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific.I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe).I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH.True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python.cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB.That is also a competing workaround for paths with spaces.And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does).Personally I find cmd/bat among the worst programming languages ever and would rather not write it.jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows.That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it.Python is a very decent programming language that is very portable across "all" systems.Perl would be the next best choice, though..I just don't like much.sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win.I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself.It shouldn't be all that hard. Modula-3 still needs a C linker.And there is C code that I didn't put in -- hand.c and dtoa.c.If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning".You pretty much must chose one.The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy>>> Jay Krell 12/31/2008 11:52 AM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/12/31 11:52:08Modified files:cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message:Partly kinda sorta fix some cross build scenarios, withoutaffecting native builds.It's a larger more general problem though.- BUILD_DIR does not necessarily equal HOST or TARGET,because of how I structured I386_CYGWIN to be a "configuration"where TARGET is still NT386.- a cross build can run the shipped binary anyway,and probably should (I didn't have the unshipped binaries around)- There should probably be automation to ensure all the tools are build.ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages,for the sniffed native platform.Cross builds have other problems.I keep hitting the following annoyances:ignoring foo/bar override when foo\bar already overriddenoverride paths should either be canonicalized to one slash typeor at least when there is a duplicate, canonicalize then and onlycomplain if canonicals varyThis is due to mixing I386_NT and I386_CYGWIN.Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake.Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due toits apparent inadequacy.The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependentthe gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcclib should probably have "target" or "build_dir" under itand/or hand.obj specifically should be merged into m3core.libThe mentor quake code generally fails in cross environments, probably due to slashes.host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violatingI'm also highly suspicious of all this "override" stuff.It clearly causes too much duplication and distribution of information.I shouldn't have to know the directory structure of my dependencies,and even if I do, I should be able to share that knowledge with their manyother dependents. The "scripts" directory also figures this sort of stuff out automatically..Being able to have multiple package stores is well and good.I'm not sure they need to ever be used in an "unshipped" form, but instead justuse alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 19:51:42 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 18:51:42 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <20090101102535.GB20605@topoi.pooq.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> Message-ID: > It is not hard to write a linker in Modula-3. By all means then, do it.> What's hard is making it compatible with existing proprietary linkers > and loaders that are poorly documented and subject to change without > notice. Good point -- do you want to be able to debug with Visual Studio or cdb/ntsd/windbg? You are in for more work then. I don't think it as ever-changing as claimed, but it isn't easy either. For Windows, you should be able to write the symbols to the .exe or a separate .dbg file, rather than deal with the truly undocumented and changing .pdb file format (there is a third party book that mostly documents it). Do the other systems change a lot? If you look across a wide range of time, well, then, yeah, you get to handle the various COFF to ELF transitions, and Apple's rewrite every few years (68K => XCOFF/CFM/PEF => MachO). They are also largely unavoidably system specific, so writing it just once won't suffice. You know..you have to learn about and handle object file formats, at least three of them, and executable file formats, again at least three -- ELF, (MS/PE)COFF, and Mach-O. And then for debugging, there is CodeView, stabs, DWARF, and maybe others. If you only want gdb, then you can just stick with stabs. (Other options seem to crash cm3cg -- like when I fiddle around with -g vs. -ggdb vs. -gstabs.) Or you can just debug with printf and have no symbols...ugh... - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:05:26 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:05:26 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B6B87.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <20090101102535.GB20605@topoi.pooq.com> <495B6B87.1E75.00D7.1@scires.com> Message-ID: > All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary > distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I am curious if anyone wants it too, but it is not difficult. Just run the vcvars32, set USE_DELAYLOAD=0 for versions prior to roughly 6.0 or 5.0spsomething (Modula-3-ism, and if anyone really wanted, could just remove it and back it just work always), and run the existing automation. It should just work. You don't have to be running on NT4, there isn't the global /usr/include business as on Unix. The headers and libraries are part of tools and not part of the operating system. Even using the latest toolsets might just work. The compiler and linker continue to output very generic stuff. The main area that platform dependencies creep in is the C runtime. For example, IsDebuggerPresent() is not on NT3.51 or Win95, but is used in newer C runtime. Some of the Interlocked functions as well, but you can also just use intrinsics and inlines for them, and then worry about the procesor and not the operating system. e.g. some of the Interlocked intrinsics/inlines require a 486, and some require something even newer -- the InterlockedCompareExchange64 stuff, if we have any. Even so, that's probably on all processors under 10 years old. XP requires it I think, but NT4/Windows 2000/Win9x did not. I think 486 is required for the modern variants of InterlockedIncrement/Decrement -- which return the entire new value, and not merely <0, 0, >0. InterlockedSomething is actually a small issue. The current C runtimes use InterlockedSomething that aren't exported on older (like pre-Win2000) systems. InterlockedIncrement/Decrement..kinda scary..you know, the functions changed meaning, in a compatible way, just from a binary's imports, you can't tell if it requires the newer meaning or if the older meaning suffices. To be clear, the newer meaning satisfies the older contract, but the older meaning doesn't satisfy the newer contract. Anyway, the modern way is to inline/intrinsic them and require a 486. But Modula-3's use of the C runtime is very small. And I dare say..we should try to remove it entirely... - Jay Date: Wed, 31 Dec 2008 12:59:03 -0500From: rcoleburn at scires.comTo: m3devel at elegosoft.comSubject: Re: [M3devel] [M3commit] CVS Update: cm3 Hendrik: I agree it is useful for those accustomed to Unix to be able to work in the Cygwin environment on Windows. I have no issue with supporting this. I assert that it is also very useful for non-Unix, Windows folks to be able to work with Modula-3 in Windows without needing to know anything about Unix, or using any Unix tools to make Modula-3 work. (BTW, I work in both the Unix and Windows worlds, but primarily in the Windows world lately.) So yes, I suppose we do need both the native Windows and Unix-compatibility implementations for Modula-3 on Windows. However, I think it important to support Modula-3 natively on all platforms rather than *requiring* a compatibility-implementation. Thus, I would always vote for having a native implementation first, then add a compatibility one later, if desired. So, from my vantage point (and others may disagree), the native Windows implementation of Modula-3 is the number one priority on the Windows platform. Cygwin is a "nice-to-have" for the Unix crowd, but not essential. I don't know much about Wine, so won't comment on it. As for taking your Modula-3 programs that run on one platform, and building and running them on other platforms, I think this is one of the best features of Modula-3. I routinely do this. Indeed, I've tried very hard to make it possible for all my programs to be both buildable and runable on both Unix and Windows platforms without the need for source code modification. I've also tried to make them interoperable across platforms, e.g. clients on one platform type, servers on another, etc. Network Objects and Pickles have been my friends here. As for the free Visual Studio stuff not working on old versions of Windows, well yes that is a problem for those old platforms, but then, they are old and Microsoft has dropped support for them long ago and continuing to support them both from a hardware and software standpoint is going to get more problematic as time goes on. It's like my 13-year old car--the last time I had it repaired the shop had to put out a nationwide search for the parts. At some point, I'll have to give it up. I believe you can use some old versions of Microsoft Visual C (e.g., v6) for these old platforms, but those were not free. I recall having to buy Microsoft C v6 when I bought CMass Reactor v4.1 many years ago. The v6 C was pretty common, so you can probably find it around somewhere today on the used market. Indeed, I still have my v6 C disks. If you or someone else wants to build a linker for Modula-3 on Windows, that's fine with me. Until then, we are stuck with what we have--a free download of Visual Studio. As for going forward, I believe we should strive to support current and future platforms more than working to add support for old ones. Of course, as we move forward, it would be nice to try to maintain support for platforms as they age, but not at the expense of supporting newer stuff. This reasoning is based on trying to keep the language alive, useful, and attractive to new programmers. If we live in the past, Modula-3 will wither away as the current crop of enthusiasts age and pass on. All that said, let me ask this question: Would it be useful to anyone if I tried to put together a binary distribution of Modula-3 for the WindowsNT 4.0 (circa 1996) platform? I still have one of those old beasts and it does still work. I suppose I could install my v6 C on it and try to build the current cm3 sources to get a binary distribution I could provide on the website. If anyone wants this, send me an email. If I get enough votes, I'll try to do it. (Of course, you will need a linker on your NT box to be able to build anything new. Not sure that I would be allowed under the license agreement to distribute the Microsoft linker from the v6 C.) (Also, my recollection [and I'm stretching to remember] is that back when cm3 v4.1 came out, it was reported to work on NT4.0, but not earlier versions of NT; it also seemed to "mostly work" on Win98 and could sometimes be coaxed into working on Win95 [I think you had to disable the garbage collector for Win95, maybe even Win98]. I know that I successfully used cm3 v4.1 on both WindowsNT 4.0 and HP-UX back then.) Regards, Randy>>> 1/1/2009 5:25 AM >>>On Wed, Dec 31, 2008 at 10:28:37AM -0500, Randy Coleburn wrote:> > On another note, All this CYGWIN stuff may be a nice way for die-hard > Unix fans to run Modula-3 on Windows, and I have no objection to > providing it, as long as it does not compromise the native Windows > implementation.It is useful to have a way to take Modula 3 programs from Unix to Windows with minimal change. That said, Modula-3 is a system programming language, and it should be possible to program in a system-dependent way. Do we need two Windows platforms, one native and one to run on a Unix-compatibility layer? And while we're at it, do we need two Unix platforms, one native and one that runs via Wine?> My main concern is the native implementation of > Modula-3 on Windows. My preference would be to keep the NT386 > implementation's dependencies on other stuff to a bare minimum, i.e., > don't introduce anything that would require someone to have to acquire > something besides what comes in the standard Windows OS in order to > make Modula-3 run. As it is now, we already have to get a C compiler > and linker. Fortunately, Microsoft has made the Visual Studio Express > editions a free download, so this is not too bad.Except that the free download won't work on old versions of Windows.This is the main reason why I have been unable in the past to use Modula 3 on Windows. At the moment, though, an overriding reason is that I have no Windows machines available.> I don't want to have to install CYGWIN either in order to make the > native implementation work on Windows. I also still prefer the > CMINSTALL, CMD, or BAT files for install as opposed to having to get > Python or something else. Just my 2 cents.> > Finally, I would go a step further and suggest that the Modula-3 > implementation on every platform should strive to require minimal > dependencies on anything not provided standard with that platform's > operating system.> > Call me an idealist, but it just galls me that I have to have a C > compiler/linker to build Modula-3. Modula-3 is a systems programming > language. It should stand on its own.It is not hard to write a linker in Modula-3.> From a purely economical > viewpoint, why should I have to buy something I don't want (C language > development environment) in order to have the privilege of using what > I do want (Modula-3 language development environment).What's hard is making it compatible with existing proprietary linkers and loaders that are poorly documented and subject to change without notice.> > Regards,> Randy-- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:14:02 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:14:02 +0000 Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu? In-Reply-To: <495B4A7B.6010505@gmx.de> References: <495B4A7B.6010505@gmx.de> Message-ID: Mostly yes. There is not always a full assortment of runtime libraries or development/static libs, like on a minimal Debian system. In your config file, make sure to put -m32 on the cm3cg invocation, and perhaps -32 or --32 on the as/gas invocation. They may or may not help. Or hurt. (They can hurt in that older builds don't understand them and error right way. They can help if you have biarch tools that default to 64bit. They make no different on modern tools that are only 32bit or biarch tools that default to 32bit). Or just try config files. Though I don't test this out much. I can look into it soon..late next week at the latest. > Hello.ms:21: Error: suffix or operands invalid for `push' Can you show us Hello.ms line 21, or the entire Hello.ms file? There is also a "native" AMD64_LINUX port that you might try. "biarchness" is nice and convenient, to kind of get two systems in one, but it can also be confusing and easily misused. OpenBSD is interesting here -- never any biarchness, simpler. - Jay> Date: Wed, 31 Dec 2008 11:33:31 +0100> From: roland.illig at gmx.de> To: m3devel at elegosoft.com> Subject: [M3devel] Is Modula-3 available on x86_64-unknown-linux-gnu?> > Hi,> > when trying to build the simplest program, examples/hello, I get the> following error messages:> > $ ~/usr.local/packages/cm3-5.4.0/bin/cm3> --- building in LINUXLIBC6 ---> > new source -> compiling Hello.m3> Hello.ms: Assembler messages:> Hello.ms:21: Error: suffix or operands invalid for `push'> Hello.ms:23: Error: suffix or operands invalid for `push'> Hello.ms:27: Error: suffix or operands invalid for `pop'> ...> > Is there some way of running cm3 on an x86_64 system?> > Roland> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 20:24:07 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 19:24:07 +0000 Subject: [M3devel] variations of waitpid..? In-Reply-To: <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> References: Your message of "Wed, 31 Dec 2008 07:13:02 GMT." <200812310722.mBV7Mftp025967@camembert.async.caltech.edu> Message-ID: You mean, this function is easy to misuse? > People who declare their own <*EXTERNAL*> Like waitpid exposed from m3core? waitpid is already easy to misuse, on a userthread system, leading to possible (though I think rare) deadlock. It is easy to misuse on pthreads, lead "just" to bad performance, and in fact I believe cm3 is doing this, via sysutils. This at least guides you between two patterns of use, and fix the perf of cm3/sysutils. On a userthread system, waitpid(pid, flags = 0) waits for the child process, with all parent threads suspended. Generally I doubt the child depends on parent threads progressing, but, yeah, that could deadlock, like if a parent thread is waiting to a file or stdin of the child, or reading a child's stdout. The various uses do waitpid(pid, flags = nohang) and then sleep and try again. pthreads just uses waitpid(pid, flags = 0) and all threads keep running. Win32 just uses WaitForSingleObject(process handle, timeout=infinite), and again all threads keep running. Just another problem with user threads, where pthreads and Win32 are fine. > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN? WaitPROCESS always yields. It is WaitPID that may or may not. So I renamed the function to DoesWaitPidYield() and moved it to SchedulerPosix.i3. ok? - Jay> To: jay.krell at cornell.edu> CC: m3devel at elegosoft.com> Subject: Re: [M3devel] variations of waitpid..? > Date: Tue, 30 Dec 2008 23:22:41 -0800> From: mika at async.caltech.edu> > > I don't think it should be possible to stop other threads if the> runtime could even conceivably support an environment where that's> not necessary. You're introducing dependencies that will lead to> (undetectable) deadlocks. People who declare their own <*EXTERNAL*>> things are, as usual, on their own.> > Mika> > Jay writes:> >> >Hm. How about:> > > >PROCEDURE Scheduler.DoesWaitProcessYield() : BOOLEAN?> > > >?> >true for pthreads/win32/cygwin> >false for userthreads> > > >duplicated privately in sysutils to avoid depending on newer m3core> > like I did for the portable C-wrapped waitpid> > > >?> > > >That way..the other code can do whatever it already is doing,> >no need to come up with abstractions for looping with nohang,> >vs. that plus the status loophole.> > > >The older fix I put it can be shrunk.> > > >The name of this is debatable, but I think the minimal level of abstraction is> > goodness.> > > >The name is actually perhaps too..complicated..like, what does it mean?> >Who uses it? What do they do with it?> > > >It means that waiting on a process to end allows threads in the waiting proces> >s to run.> >Perhaps it is too low level.> > > > > > - Jay> >> >> >> >----------------------------------------> >> From: jay.krell at cornell.edu> >> To: m3devel at elegosoft.com> >> Date: Wed, 31 Dec 2008 05:33:22 +0000> >> Subject: [M3devel] variations of waitpid..?> >>> >>> >> variations of waitpid..> >>> >> Hey, we already have, that I put in on my initiative, months ago:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\POSIX\ThreadPosix.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are the same here except ThreadPosix.> >m3 calls Pause(). *)> >> VAR> >> result: int;> >> statusM3: Uwaitpid.waitpid_status_t;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := Uwaitpid.waitpid(pid, statusM3, Uwaitpid.WNOHANG);> >> IF result # 0 THEN EXIT END;> >> Pause(Delay);> >> END;> >> 0 *>> >> RETURN statusM3.w_Loophole;> >> END WaitProcess;> >>> >> pthreads and cygwin:> >>> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\WIN32\WaitProcessCygwin.m3> >> and> >> C:\dev2\cm3.2\m3-libs\m3core\src\thread\PTHREAD\ThreadPThread.m3> >>> >> PROCEDURE WaitProcess (pid: int): int => >> (* ThreadPThread.m3 and ThreadPosix.m3 are very similar. *)> >> VAR> >> statusM3: Uwaitpid.waitpid_status_t;> >> BEGIN> >> LOOP> >> WITH r = Uwaitpid.waitpid(pid, statusM3) DO> >> IF r> 0 THEN> >> RETURN statusM3.w_Loophole;> >> END;> >> < 0*>> >> END;> >>> >> END;> >> END WaitProcess;> >>> >> win32:> >> PROCEDURE WaitProcess (pid: int): int => >> BEGIN> >>> >> RETURN 0;> >> END WaitProcess;> >>> >> ThreadPThread and WaitProcessCygwin should share code, but that's not the po> >int.> >>> >> However, now, to fix sysutils, we need something LIKE:> >>> >> UNSAFE INTERFACE SystemPosixWaitPid;> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int;> >>> >> BEGIN> >> END SystemPosixWaitPid.> >>> >> pthreads:> >>> >> UNSAFE MODULE SystemPosixWaitPidEfficient EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> BEGIN> >> RETURN waitpid(pid, status, 0);> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidEfficient.> >>> >> user threads:> >>> >> UNSAFE MODULE SystemPosixWaitPidPause EXPORTS SystemPosixWaitPid;> >>> >> FROM Ctypes IMPORT int;> >> FROM Sysutils_Uwaitpid IMPORT waitpid_status_t, waitpid;> >> FROM Utypes IMPORT pid_t;> >>> >> PROCEDURE WaitPid(pid: pid_t; VAR status: waitpid_status_t): int => >> VAR result: int;> >> CONST Delay = 0.1D0;> >> BEGIN> >> LOOP> >> result := waitpid(pid, status, Sysutils_Uwaitpid.WNOHANG);> >> IF result # 0 THEN> >> EXIT> >> END;> >> Thread.Pause(Delay)> >> END;> >> END WaitPid;> >>> >> BEGIN> >> END SystemPosixWaitPidPause.> >>> >> questions:> >>> >> Aside: Where do I put "unsafe"?> >>> >> Relevant: Sysutils has to implement this itself, right?> >> Due to bootstrapping concerns.> >>> >> But we should still put /something/ in m3core, right?> >>> >> So then the big question, what to call it?> >>> >> Probably, the existing WaitProcess that does more than this WaitPid, should > >be scaled> >> back and only do what WaitPid does?> >>> >> I will likely commit exactly what is shown above.> >> The new interface and modules are in sysutils. The interface is private.> >>> >> The unfinished part is putting something in m3core, that sysutils could use,> > if> >> not for boostrapping issuers, that it could use in the future.> >>> >> I kind of thing something a bit more is needed.> >> You know, the Win32 implementation that asserts(false) is suspicious.> >> At the very least, it should be removed.> >>> >> I suspect I am missing something though, as suggested by Win32 having an ass> >ert(false) implementation.> >> Either that needs to be removed, or somehow the code can be unified, using j> >ust the existing m3core/libm3> >> interfaces?> >>> >> I also suspect part of the problem is that sysutils implements to a large ex> >tent> >> the same abstractions as m3core and/or libm3, but with variations, causing i> >t to be unable> >> to build on what is there, but having to reinvent..which suggests the m3core> >/libm3 abstractions> >> maybe are not general/featureful enough.> >>> >> (Btw, unstated above, is that sysutils/m3makefile will have the same sort of> > switch as m3core does, wrt if it builds for user/alaram threads or kernel/pth> >reads.)> >>> >>> >> ??> >>> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:30:40 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:30:40 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> Message-ID: <495B901F.1E75.00D7.1@scires.com> Jay: First, I do want to say thanks for all you are doing for the cause of Modula-3. I don't think we say thanks enough for what others do for us. THANK YOU! I don't want to be perceived as "complaining", rather I am trying to voice my opinion in the hopes of influencing future direction. Of course, since I'm not doing the work, I can only make suggestions. In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. Now, I'm not sure how much effect this deoptimization has (maybe little), but it is clear that in this case your implementation choice has favored the non-native implementation over the native one. IMO, tradeoffs of this type are not good. If one is trying to convince someone to use Modula-3, why would you want to give them a "deoptimized" version just to make it easier to support a non-native environment---indeed, one that they may not even want/use? If we have to make a trade-off, I say favor the native implementation always over the non-native one. What I'm trying to say here is that my experience as a software, systems, and program engineer is that I've always been forced to justify the cost/benefit of development tools for any project. Many times I've had to go head to head with folks higher up in my own organization or in the customer's organization whose preconceived opinions were based on rumor and what they've heard rather than actual factual hands-on experience. I want to pick the best tool for the job instead of being forced to pick an inferior tool because someone higher in the food chain demanded it based on faulty preconceived opinion. I like Modula-3. I've found that I am more productive using it. But, if the compiler doesn't produce efficient code, I lose ground in arguing with the higher-ups. As for Python, I've never ventured to learn it, so for me, it is something of a mystery. But you miss the point. I'm not arguing the merits of Python, I'm just saying that anything Modula-3 requires on top of what is provided by the standard host platform represents a potential obstacle or barrier to ease of use/implementation. It also sends the message that somehow Modula-3 is not complete on its own, we have get Python just to install and oh yes we need a C compiler and a linker and a ...? IMO, ultimately we need a turn-key download and install routine for Modula-3 that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install, I can run it on Windows, but if you give me an install routine that requires I first install something else, e.g. Python, then that becomes a barrier to the folks who don't know Python or already have it installed. Am I alone in this line of thought? If so, I'll just be quiet. BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. My problem is that right now, it is hard to understand all the dependencies for proper building of cm3 from scratch. Indeed, I've contributed some CMD/BAT install stuff in the past, but it is now out of date. Perhaps if there is a way we could better document the proper build order and dependencies, it would be easier for folks in the community to help in this area. I certainly would volunteer to do the CMD files as long as I had enough information to do it right. At one point, I thought we had a file that showed the package build order and dependencies. Not sure if the format of this file is well-documented or if the file is kept up-to-date. Regards, Randy >>> Jay 12/31/2008 1:34 PM >>> > you've made the CYGWIN implementation "appear" > be the same as the native Windows implementation. > But they are not the same. But they mostly are, from the front end's point of view. They are both little endian, x86, use the same jump_buf size (I grew it to match Cygwin's, which is a deoptimization of stack use on native NT386), the same type sizes and alignments..er..not sure about LONGINT. They might vary in newline and one uses the internal backend and one the external, however which backend they use is actually minor to the front end, though I think it affects if the front end "deals with" calling conventions, particularly left-to-right vs. right-to-left and struct returns. Object code produced by one can generally be linked with object code produced by the other. SOLgnu and SOLsun are a better example of this, though they do go ahead and duplicate tons of stuff that if I had implemented them, I would have avoided. They are truly identical in every respect in the front end and back end. They only vary in the config file. This is wasteful, because, "by default", if I'm put together a comprehensive cross build system without being a little smarter, I end up building two identical m3cgs. My config files are willing to "probe across" SOLsun and SOLgnu though, so I need only one m3cg for the two of them. Given that "NT386" can describe a combinatorial explosion of configurations, it makes some sense to keep it just one if possible. The compiler mostly doesn't care. > "- BUILD_DIR does not necessarily equal HOST or TARGET, > because of how I structured I386_CYGWIN to be a "configuration" > where TARGET is still NT386." > IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". BUILD_DIR also does not equal HOST or TARGET when doing profiled builds, which admittedly I never have. Therefore, BUILD_DIR is arbitrary. I might be confusing something here though -- since I have never built a profiled build, I also haven't shipped one. It could be that "shipped BUILD_DIR" does equal TARGET, for profiled builds, in which case I did set a precedent. But the "override" code isn't using shipped files, so.. I have to check. > as long as it does not compromise the native Windows implementation Agreed and it basically doesn't. Show me where it does. The only thing I can think of is the jmpbuf size. > I don't want to have to install CYGWIN either in order to make > the native implementation work on Windows. Please don't complain about stuff that isn't true or without being more specific. I know of no dependency by the native implementation on Cygwin. In fact, installing Cygwin does tend to break the native implementation, depending on $PATH, because it has a link.exe that is a completely different program (ie: ln.exe). I tried experimenting with using cl to drive link.exe but couldn't quite get it to work, for stupid reasons related to response files, which surely is fixable by extending response file support in cm3... As it is, I usually delete \cygwin\bin\link.exe, or remove cygwin from $PATH. True, I don't ever uninstall Cygwin for testing, so the dependency could creep in by accident. > I also still prefer the CMINSTALL, CMD, or BAT files > for install as opposed to having to get Python or something else. Just my 2 cents. Once built and installed, there is no dependency on cmd, bat, cminstall, or python. cminstall is pretty worthless imho, as long as you set PATH, INCLUDE, and LIB. That is also a competing workaround for paths with spaces. And allows moving around among different versions of Visual C++, which is good and bad. Either you can have n installs of cm3, each configured tightly for one specific Visual C++, or you can have 1 install of cm3, that is flexibly configured, that you "reconfigure" by altering the environment. I do the second. If you want to build the system, in a well automated way, with cmd and bat, you are welcome to write them. Maybe the ones I left in the tree work, but i never use them any longer. I use the Python all the time. Or you can manually cd around (as I suspect Tony does). Personally I find cmd/bat among the worst programming languages ever and would rather not write it. jscript via cscript.exe would be a good alternative, it is always there at least as of Windows 2000 or perhaps with IE installed on older versions, but then you have to duplicate work across Unix and Windows. That is, for Windows-only no-cmd, no-install command line automation, JScript and VBScript are good choices, but Windows-only rules them out. The install is worth it. Python is a very decent programming language that is very portable across "all" systems. Perl would be the next best choice, though..I just don't like much. sh/bash requires an install on Windows, so its built-inness on Posix I claim isn't a conclusive win. I strongly encourage you to try out Python. Another avenue is to merge the sh/cmd/python into cm3 itself. It shouldn't be all that hard. Modula-3 still needs a C linker. And there is C code that I didn't put in -- hand.c and dtoa.c. If someone writes a linker in Modula-3, or gets the .obj loader working, I'll be more open to eliminating C. But using C is much less error prone and much more portable, in some parts of the system. You may label C as "evil", but the other "evil" I am battling is tedious error prone "header cloning". You pretty much must chose one. The header cloning can be reduced, and its error proned-ness and difficulty various per system, depending on how gnarled up the headers are with ifdefs, compatibility hacks, processor-specificty, "cleansing" (where the installed headers are processor specific, deriving from #ifdef'ed or somesuch other files, but now I argue both sides, #ifdef is hard to read, but removing them removes information unless you track further back), etc. For example the OpenBSD headers are much more readable. I suspect NetBSD are too but haven't looked yet. The Linux headers contain a lot of #ifdef and they are also processor-specific I think -- you know, my /usr/include/errno.h on one system I think didn't show that x86 and SPARC use different values. - Jay Date: Wed, 31 Dec 2008 10:28:37 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: Please explain what you mean by: "- BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386." IMO, it would be wrong to change the meaning of things like "BUILD_DIR", "HOST", and "TARGET". Indeed, I have stuff that depends on the meaning of these things. Your statement seems to imply that "under the covers" you've made the CYGWIN implementation "appear" to be the same as the native Windows implementation. But they are not the same. On another note, All this CYGWIN stuff may be a nice way for die-hard Unix fans to run Modula-3 on Windows, and I have no objection to providing it, as long as it does not compromise the native Windows implementation. My main concern is the native implementation of Modula-3 on Windows. My preference would be to keep the NT386 implementation's dependencies on other stuff to a bare minimum, i.e., don't introduce anything that would require someone to have to acquire something besides what comes in the standard Windows OS in order to make Modula-3 run. As it is now, we already have to get a C compiler and linker. Fortunately, Microsoft has made the Visual Studio Express editions a free download, so this is not too bad. I don't want to have to install CYGWIN either in order to make the native implementation work on Windows. I also still prefer the CMINSTALL, CMD, or BAT files for install as opposed to having to get Python or something else. Just my 2 cents. Finally, I would go a step further and suggest that the Modula-3 implementation on every platform should strive to require minimal dependencies on anything not provided standard with that platform's operating system. Call me an idealist, but it just galls me that I have to have a C compiler/linker to build Modula-3. Modula-3 is a systems programming language. It should stand on its own. From a purely economical viewpoint, why should I have to buy something I don't want (C language development environment) in order to have the privilege of using what I do want (Modula-3 language development environment). Regards, Randy >>> Jay Krell 12/31/2008 11:52 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:52:08 Modified files: cm3/m3-comm/netobj/src/: netobj-ov.tmpl cm3/m3-comm/sharedobj/src/: sharedobj-ov.tmpl cm3/m3-libs/libm3/src/bundleintf/: bundle-ov.tmpl cm3/m3-ui/zeus/src/: m3zume-ov.tmpl cm3/m3-ui/juno-2/juno-app/src/: m3makefile Log message: Partly kinda sorta fix some cross build scenarios, without affecting native builds. It's a larger more general problem though. - BUILD_DIR does not necessarily equal HOST or TARGET, because of how I structured I386_CYGWIN to be a "configuration" where TARGET is still NT386. - a cross build can run the shipped binary anyway, and probably should (I didn't have the unshipped binaries around) - There should probably be automation to ensure all the tools are build. ie: do-cm3-tools or do-cm3-crosstools. ie: build just the needed packages, for the sniffed native platform. Cross builds have other problems. I keep hitting the following annoyances: ignoring foo/bar override when foo\bar already overridden override paths should either be canonicalized to one slash type or at least when there is a duplicate, canonicalize then and only complain if canonicals vary This is due to mixing I386_NT and I386_CYGWIN. Similarly the problem demonstrated regarding m3unix.h in Uwaitpid.quake. Perhaps the change I made to allow forward slashes on Win32 was not good, esp. due to its apparent inadequacy. The "lib" directory, specifically \cm3\lib\hand.obj is target..er, configuration dependent the gcc hand.obj cannot be directly linked by Visual C++, for reasons to do with libgcc lib should probably have "target" or "build_dir" under it and/or hand.obj specifically should be merged into m3core.lib The mentor quake code generally fails in cross environments, probably due to slashes. host=NT386 (GNU?), target=LINUXLIBC6 m3zume is access violating I'm also highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Wed Dec 31 21:39:54 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Wed, 31 Dec 2008 15:39:54 -0500 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: References: <20081231105714.24CF710D5DDE@birch.elegosoft.com> <495B4F55.1E75.00D7.1@scires.com> Message-ID: <495B9248.1E75.00D7.1@scires.com> Jay: I was just trying to explain a bit about the history of and reason for the windowsResources package. As for m3overrides, I rarely use it anymore. At one point in time, we had a large project going on and the engineers used m3overrides for local testing before shipping to the main repository. That way, we avoided committing something that was obviously broken and causing other engineers to have downtime. I also seem to recall devising a neat way to use m3overrides to build against stubs and to create test editions of a module. Alas, it's been a long time since I've managed a project with many Modula-3 engineers working concurrently. Regards, Randy >>> Jay 12/31/2008 1:44 PM >>> "some builds" is builds using overrides -- "buildlocal". You can often sleaze by by having a compatible already shipped libm3, but you aren't supposed to do that. You are supposed to be able to make breaking changes to libm3 and keep them temporarily separate from your mainline in-use package store. I'm highly suspicious of the override mechanism, but this is how things are done throughout the tree and I'm not going to change it any time soon. This merely brings windowsResource inline with how everyone else does it. I understand there is a need for multiple package stores, so you can develop and experiment and break one, and then go right back to a working one, but I suspect alterate package store should be "shipped to other roots", not the unshipped output sitting in the source tree. "Fixing" this would allow more concurrent builds. Basically, you know, there are three sets of outputs: temporary files that aren't going to be shipped -- .o files files that aren't going to be shipped -- executables/.dll/.so -- in the same place as the .o files the shipped files The first two sets are in the source tree, but I think should not be. And then, if that is fixed, it might be ok to remove the second set and just ship things as they are built, right to the third set. You know..this is one of those ways in which Modula-3 was arguably ahead of its time, but the world has caught up, and possibly surpassed it. In particular, if you only look at a single package, Modula-3 has always kept the source tree clean, it has always and only supported putting the outputs outside the source tree. Good. These days, most packages outside of Modula-3 support either way (though some only work one way or the other, or are only tested one way or the other; good example is that gcc only really supports out-of-source builds). Where Modula-3 fails though is if you consider multiple packages. Then, in a real sense, the outputs are in the source tree. The analog outside the Modula-3 world is..tenuous..usually there is rarely automatic stitching together of multiple packages. In fact, even Modula-3 -- cm3 -- doesn't do that. But we have heavily used automation that does (scripts directory). The only analog I can think of in the non-Modula-3 world is how the gcc tree stitches together multiple packages, and keeps all the outputs outside the overall source tree, instead of only on a package by package basis. - Jay Date: Wed, 31 Dec 2008 10:58:44 -0500 From: rcoleburn at scires.com To: m3devel at elegosoft.com Subject: Re: [M3devel] [M3commit] CVS Update: cm3 Jay: I'm not sure why an m3overrides file is needed for "some builds". The WindowsResources package simply adds a template that allows folks on Windows platforms to include icons and other resources in their programs. This package shouldn't be needed for non-Windows platforms. Indeed, the template (winRes.tmpl) uses "if equal (OS_TYPE, "WIN32")" to bracket all of its activity, so it shouldn't do anything on non-Windows platforms. Perhaps, the same logic should be used in the m3makefile to prevent inclusion of the WindowsResource package on any platform other than Windows, but then that would probably force similar logic in all packages that import WindowsResources. I know that CM3IDE uses WindowsResources and, of course, I have a number of programs that use it. Regards, Randy >>> Jay Krell 12/31/2008 11:57 AM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/12/31 11:57:14 Added files: cm3/m3-sys/windowsResources/src/: m3overrides Log message: Add missing override to fix some builds. I'm highly suspicious of all this "override" stuff. It clearly causes too much duplication and distribution of information. I shouldn't have to know the directory structure of my dependencies, and even if I do, I should be able to share that knowledge with their many other dependents. The "scripts" directory also figures this sort of stuff out automatically.. Being able to have multiple package stores is well and good. I'm not sure they need to ever be used in an "unshipped" form, but instead just use alternate roots and create new empty roots as needed. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Dec 31 21:55:13 2008 From: jay.krell at cornell.edu (Jay) Date: Wed, 31 Dec 2008 20:55:13 +0000 Subject: [M3devel] [M3commit] CVS Update: cm3 In-Reply-To: <495B901F.1E75.00D7.1@scires.com> References: <20081231105208.9CF4710D5DDE@birch.elegosoft.com> <495B4846.1E75.00D7.1@scires.com> <495B901F.1E75.00D7.1@scires.com> Message-ID: > In reading your post, you state that you deoptimized the native implementation to make it match Cygwin. It is such a slight deoptimization, I think it is ok. Are you comfortable with only slightly optimizing never inlining integrated backend? Not optimal. But it sure does build faster than gcc. Great for productivity. And its "always only slightly optimizing" mode generates much better code than gcc's default "no optimization at all" mode. But gcc with optimizing should be able to beat it handily -- e.g. inline. Do you use gcc's optimizer or not? I usually don't. I want to be able to debug line by line with a close correlation of binary to source. I have plenty of experience debugging optimized code, and it can be an interesting fun educational challenge, but unless I'm really bored, I'd rather not do it. If you don't use the optimizer, then again, not optimal. I wouldn't mind /some/ optimization, esp. some inlining and constant propagation, but I dislike it when the debugger jumps around with no clue as to what "line" of code I am on (because "lines" of code often don't exist in optimized code, a line can go away, or get split up, or become shared with other "lines", etc.). Are you aware of the cost of PushEFrame/try/raise/finally in Modula-3? Nobody seems to care much. Granted, SOLgnu/sun does it better, so people care.Almost anything is more efficient. Windows/x86 C/C++ does something similar but way more optimized. All other C/C++ systems use "stack walking" and incur little cost from the "try" alone. Though raise is often more expensive. Ok, granted, my deoptimization is to make PushEFrame even less efficient. It is already space (stack) and CPU inefficient. I made it more space (stack) inefficient. Could contribute to stack exhaustion. > IMO, ultimately we need a turn-key download and install routine for Modula-3 > that just works, out-of-the-box. If you give me an EXE, a CMD, or a BAT, for the install Can you try the uploaded archives I produced? Are they really too difficult to use?You extract them, optionally rename the root directory, run vcvars, add cm3 to %PATH%, and are done. Arguably much better than the older interactive cminstall. Not sure about the non-interactive variant. I understand your point. No Python is needed to download and run. No Python is really needed to build the system, but cm3 itself only builds a package at a time. The sh and Python automation is around cd'ing around and building, in the right order, with some filtering, and my Python adds ability to pick target platform on the command line, without messing around every time with config files and %PATH% and such. It is nice and convenient, it adds features that cm3 doesn't provide, but you don't have to have it. > BTW, I can certainly help in maintaining the CMD/BAT install routines or in making a better CMINSTALL. Your cmd/bat are actually still probably correct or very nearly so. The dependency orders rarely/never change. The main change is that sysutils was added. And perhaps that the dependency order became "more important" but not actually "different". That is, probably for a long time, an old cm3 could compile a current m3core/libm3. Now you need a cm3 with LONGINT support to build them (which itself is now a few years old). So you must first build the compiler with the old compiler, THEN build the current runtime. The "upgrade" scripts handle this. Again you can just cd around in the right order, with some repetition (you build the compiler first against the old runtime, then build the current runtime, then rebuild the compiler against the new runtime; arguably not needed but very much what you want to do). [redundancy here due to bad editing, sorry] Also, if you are bootstrapping from an older build, then you use their m3core and libm3, and then build the compiler up in dependency order starting with sysutils. If you are building from scratch with a current compiler, then you can start with m3core and go up from there. (Or import-libs on Windows, if needed, but this really just a workaround for old distributions, like 4.1, like before Tony's change to remove the "gcdefs" stuff.) > At one point, I thought we had a file that showed the package build order and dependencies. I was a big proponent of this, and it now exists and is in use..but I don't use it usually. :) It is the scripts/pkginfo.txt file. All the sh scripts use it I believe. Therefore the tinderbox uses it I believe. My cmd that I don't use (scripts/win) I think use it. My python code does not use it. Oops. Well, I think I did implement it at least, so I didn't push someone else to and then not use it.. :) (Pick your evil -- do something yourself, and then not use it, or push someone else to, and then not use it). Eh, I should just use it from the Python, not a big deal. Not only the build order, but the file also aids in package set selection. It doesn't do the filtering though, oh well. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: